By Russ Banham, Contributor
In 2000, my Aunt Isabel passed away at the age of 80. Unattached and with no children of her own, “Aunt Izzy” took a profound interest in my childhood development. A cultured woman, she gave me important books to read and took me to places my working class pals didn’t go to—the theater, ballet, and museums. She even squired me on a three-week tour through Europe when I was 11. A few days before she died from cancer, she left a heartfelt message on my clunky answering machine. I never erased that message, listening to it whenever I needed a boost.
Through a combination of audio-video recordings, speech recognition software, and machine learning, one can ask a question of a departed family member and receive a response…
Today, people needing a lift from their loved ones’ deaths have the extraordinary opportunity to not just passively listen to their old voicemails but actively engage with their virtual beings. Through a combination of audio-video recordings, speech recognition software, and machine learning, one can ask a question of a departed family member and receive a response, creating the illusion they’re still alive and well.
Not everyone will be delighted at the prospect of conversing with the ghostly vestiges of late friends and family members, perceiving it distasteful and maybe a bit creepy. This possibility, however, hasn’t curtailed a stalwart group of innovative technology and entertainment companies’ development of different platforms that provide intimate “discussions” and other experiences with people no longer with us.
YOU MAY ALSO LIKE: Haptic Happenings: How Touch Technologies Are Taking on New Meaning
The companies’ creative efforts range from interactions with late relatives and friends to the opportunity to query a former CEO whose wisdom and business chops were preserved on video before the executive passed. Most of the platforms are predicated on our all-too-human motivation to preserve and even persist our relationships with people who have transitioned into another realm of being, or nothingness, depending on your view. Other platform developers have created digital avatars that resemble actual people, books, and movie characters, and even oneself—a clone that can spend time with the kids while you’re at play or work.
“People forget the pyramids in Egypt weren’t just monuments, they were funerary complexes for people to remember the dead and reflect upon their lives,” says neuroscientist Michael Graziano, PhD, professor of psychology at Princeton University, where he heads the Princeton Neuroscience Institute. “This is just another iteration of something that has been going on for a very long time.”
Nice to Meet You, Jack
The marketing maxim of HereAfter sums up the company’s mission: “Never lose someone you love.” James Vlahos, cofounder and CEO, had a basic understanding of artificial intelligence (AI) when he learned his father was dying from stage IV lung cancer. Vlahos created a software program called Dadbot that drew from his conversations and correspondence with his father to “converse” with his avatar likeness.
Upon the person’s passing, friends and relatives can ask a question to the interactive chatbot, which replies in the interviewee’s actual voice.
HereAfter builds upon the platform, capturing a person’s oral history in a series of interviews while they are still alive and then converting it into a chatbot accessible through a smartphone. The interview may be a few hours long or consume several days, depending on the specific use of the chatbot. Upon the person’s passing, friends and relatives can ask a question to the interactive chatbot, which replies in the interviewee’s actual voice.
“Hey great-grandpa, what was it like in L.A. in the years before cars were self-driving?”
“Well, kiddo, we fried on the freeways stuck in traffic.”
That’s a simpler version of the chatbot; more extensive uses came to Vlahos’s mind when a curator at the Smithsonian Institute contacted him regarding a planned Technology of the Future’ exposition. “It got me thinking about expanding the market for HereAfter, from stories that have family interest to a bot with more universal appeal, such as the oral history of someone important in a particular field,” he explains.
That “someone” might be a renowned scientist, top athlete, or great business leader like Jack Welch, the formidable—and very much alive—CEO of General Electric (GE) in its heyday. Assuming Welch submitted to a series of interviews on his business challenges and triumphs while leading GE, this information would feed the development of a virtual bot that current and future employees could query and otherwise chat with. “Bot Jack’ would respond in Mr. Welch’s actual voice with very specific and useful advice on business situations,” Vlahos explains.
As a person’s stories are collected, they are annotated—essentially labeled as different data. Using AI-embedded word recognition software and machine learning algorithms, the person’s words and phrases in the interviews are correlated to words and phrases asked in a question. “We plan to market our first offering to families and then to prominent individuals,” Vlahos says.
“By mapping the person’s stories into a taxonomy, we can align them with questions involving similar semantics.”
—Hossein Rahnama, visiting professor at MIT Media Lab and associate professor, director of research and innovation at Ryerson University
Hossein Rahnama, a visiting professor at the MIT Media Lab, has built a platform with a similar objective called Flybits. The platform is a spinoff of his PhD research on understanding the context of large data sets and leveraging them to personalize information for an end user. “The medium of delivery can be a conversational interface like a chatbot, a voice assistant like Siri, or a set of stories or narratives,” says Rahnama, also an associate professor and director of research and innovation at Toronto’s Ryerson University.
Once stories are collected on the experiences of a person like a business leader, scientist, or an academic, they are subsequently mapped into a “decision tree”—a tree-like model of decisions and possible outcomes. “By mapping the person’s stories into a taxonomy, we can align them with questions involving similar semantics,” Rahnama says.
The decision tree grows as more experiences populate the “limbs.” “The responses of the avatar can be useful in guiding a decision but are not to be confused with an actual decision,” says Rahnama. “We see Flybits as a decision-support system, not a decision-making one.”
Use Eternime to Build a Digital Clone of Your Loved One
Another platform designed to perpetuate the lives of the departed is Eternime, co-founded by CEO Marius Ursache. Like Vlahos, Ursache was motivated to develop the platform by the death of a loved one: his grandmother. He regretted that he hadn’t spent as much time with her as he would have liked and retained only a few fond memories and photographs. Eternime was created so others could preserve fuller stories of their late loved ones.
YOU MAY ALSO LIKE: How AI Helps Give ALS Patients a Voice
To do this, the company collects data on a person’s digital footprint—words, photos, videos, texts, and SNS content captured online and through social media. An avatar is crafted from this raw material. The goal is to create a digital twin or clone, “a ‘Tamagotchi’ version of the user that learns from and grows with the user throughout the person’s life,” Ursache says, referencing the once popular electronic toy that displayed the digital image of a creature that had to be looked after by its owner like a real pet.
Upon the person’s death, children and adults could interact with the avatar. But Ursache has a more present-time ambition for Eternime. By creating a mirror version of oneself, the clone can observe one’s actions and behaviors while still alive, illuminating our personal failings and suggesting ways to improve them. It could also perform certain chores “like advising people and talking with journalists,” Ursache says. “We all have a dream of cloning ourselves to be able to do more than time allows.”
Lucy: A Virtual Being to Provide Assistance and Company
In similar vein, the entertainment company Fable Studio has created a virtual being to provide assistance, instruction, and company. The avatar’s name is Lucy, and she is based on a character with the same name in The Wolves in the Walls, a novel by Neil Gaiman that teaches children about irrational fears.
Children who have read the book strongly identify with Lucy, says Pete Billington, co-founder of Fable Studio. Billington wants to extend this relationship. “Our heritage is storytelling,” he explains. “We’re fascinated by the opportunity to create ongoing relationships with virtual characters over long periods of time who learn, grow, and teach us as we teach them.”
Wearing lightweight virtual reality (VR) goggles, a child learning a song on the piano would “see” Lucy sitting next to her on the bench. If the girl hits a wrong note or speeds up a passage, Lucy would point out the mistakes. If she plays the song well, Lucy would applaud. Over time, the relationship blossoms.
AI is the nucleus of this relationship. Fable embeds biographical information of the user into the platform (the child already is familiar with Lucy, having read Gaiman’s book). Each time the user and Lucy interact, the platform captures the experience, cultivating the relationship. The bond is further strengthened by having Lucy reach out to the user through text, email, social media, and voicemail. The company plans to release an early version of its Lucy VR experience by the end of the year. A premium version will follow.
What does Lucy have to do with the dearly departed? Billington says the same platform could be used to create a virtual clone of a family member for children and adults to communicate with. “If grandma passes away, she could come back as a hologram in avatar form,” he says. A relationship that seemingly ended persists as the user ages.
“Since you know the person is gone, you accept the virtual equivalent for what it is—a comforting vestige. There is nothing wrong or unethical about it.”
—Michael Graziano, PhD, professor of psychology at Princeton University and head of the Princeton Neuroscience Institute
Many people may blanch at the prospect of communicating with their late loved ones, but there’s nothing “inherently wrong” in wanting to keep the memories alive, professor Graziano says. “We have a vigorous culture out there of people who have died, but remain online, their photos and comments still resident on Facebook, Instagram, and YouTube,” he explains. “Since you know the person is gone, you accept the virtual equivalent for what it is—a comforting vestige. There is nothing wrong or unethical about it.”
Although I’m not a nostalgic person by nature, that gives me comfort. There might be a way to reanimate Aunt Izzy as an avatar, digitizing a compilation of old photos and home movies I took in my 20s on a video camcorder the size of a scooter. She could sit here in my office reading Agatha Christie, her favorite, while I click away on the keyboard.
Russ Banham is a Pulitzer-nominated journalist and best-selling author.