Interacting with a ‘virtual’ holocaust survivor


Those dedicated to preserving the memory of the Holocaust are facing a difficult deadline. Seventy years after the end of World War II, most of the Holocaust survivors still alive were children or teenagers during the war, and soon even they won’t be able to share their experiences directly with the rest of us.

The USC Shoah Foundation recorded 53,000 video testimonies of survivors to help preserve their stories. But educators have found that children learn best when interacting directly with a survivor, whether in a museum or a classroom.

“What we’ve seen is that the connection with that history and the bond that those kids have is completely different than if they had never met a survivor; if they watched a movie or if they just read something,” said Heather Maio, a Holocaust exhibit curator, at the Shoah Foundation’s offices.

“It’s a completely different experience to be in the presence and ask someone your own question and get a response in their own words. So we really felt it was necessary to find a way to replicate that for future generations, so it wasn’t going to be lost,” she said.

Maio has teamed with Shoah and the USC Institute of Creative Technologies (ICT) to create a new genre of film that adds a 3-D and interactive element to the survivor’s life stories. The project is called “New Dimensions in Testimony.”

The first subject is Pinchas Gutter, born in 1932 in Lodz, a survivor of the Warsaw Ghetto and six Nazi concentration camps. He lost his parents and twin sister at the Majdanek concentration camp in Poland. He was then imprisoned at Buchenwald, Colditz and finally sent to Theresienstadt, where he was liberated by Soviet troops on May 8, 1945. Gutter now lives in Toronto.

It was important, Maio said, to find a survivor with the mental and physical stamina to respond to hundreds of questions, many being only slight variations on others. For the 3-D interview, Gutter was seated on a red chair surrounded by neon green fabric, 6,000 LED lights and 53 cameras. The video is stitched together, then holographically projected to make a virtual image of Gutter that appears like a three-
dimensional person.

“It had never been done before, so we needed someone that we knew would be flexible and patient,” Maio said.

The filming took five days and resulted in about 27 hours of responses. After an initial round of focus group testing, Gutter was filmed for another two days. He was asked approximately 2,000 questions about his thoughts and experiences, ranging from the obvious (“How did you survive the Holocaust?”) to the obscure (“What happened to your best friend?”) to the absurd (“Did you meet Hitler?”).

“There’s nothing edited from the point that he speaks until he finishes. So nothing is touched or manipulated or edited in his responses. That was really important for the integrity of the interview,” Maio said.

Shoah organized focus groups to enable people to witness the technology in action, while also making the technology “smarter” as it better learns how to find appropriate answers for different ways of phrasing questions.

I attended a session on June 11, and listened as Shoah staffers and volunteers asked test questions. Gutter’s image was projected onto a large screen, not in 3-D, but it seemed as though he was listening in a remote location. He was asked whether he has nightmares, what he does for fun, what he remembers of his twin sister and whether he’s worried about anti-Semitism today. For the most part, the projected image of Gutter responded to the questions posed to him coherently and concisely.

Some answers were touching. When asked, “What were your first impressions of Majdanek?” Gutter described seeing a field of sand. When Jews got off the transport trains, they began burying their jewelry and other precious belongings in the sand. He realized later that the Nazis expected them to do that, knowing the Jews wouldn’t have the chance to return and dig up the items.

The project relies on the ICT’s development of a natural language processor, which filters the questions through speech-recognition software and then searches for the most appropriate response. The institute also lent its experience with holographic display to make the survivor look three-dimensional without requiring viewers to wear special eyeglasses.

Gutter’s 3-D image is projected holographically

But there were a few glitches in the program on the day I was there. When Gutter was asked “What was your occupation after the war?” he explained that he’s retired, is a cantor at his synagogue and volunteers to visit people at hospitals and jails. When asked the question again, he explained more directly that after the war he became a credit manager at a finance company in South Africa.

When I asked, “Do you believe in God?” he responded with an answer about the impossibility of forgiving the Nazis. When I asked again, he answered the correct question, explaining that he believes in “a being that watches over us all the time,” though he doesn’t comprehend the full nature of God.

When asked what his favorite subject was in school, he answered, “My favorite food is gefilte fish that I make according to the way that my mother used to make it.” The researchers said they hadn’t asked Gutter about his favorite subject in school.

If someone asks Gutter a question the program doesn’t understand, he’d first ask the person to repeat the question. If the program still doesn’t understand, Gutter might suggest they ask him about something else. Gutter was also allowed to skip answers that he felt were too personal or uncomfortable. If a viewer were to ask such questions, he’d simply explain that he’d prefer not to answer it. 

Maio said viewers seemed just as cautious when talking to a virtual survivor as they did in person.

“As this technology gets more familiar to the public, they’ll be more apt to ask those difficult questions that they wouldn’t necessarily even ask a survivor in person because they don’t want to upset them,” Maio said. 

The team behind the project expects to film a total of 10 survivors using this current process. Meanwhile, they expect that as the technology improves, the interactions will become even more realistic.

“I don’t think you can replace the human interaction, the way human beings look each other in the eye. …  But what we’ve tried to capture here is, what’s the essence of what’s being told?” said Stephen Smith, executive director of the USC Shoah Foundation. “And how can the content that we’ve collected from these very precious people be stored in such a way that my curiosity, when it’s sparked, can continue to draw from the wealth of their experience? And I think we’ve narrowed the gap on that a little bit.”

For more information about the “New Dimensions in Testimony” project,

+