By Kathryn Nave, Contributor
The sense of inhabiting a physical body is something most of us take for granted. Yet to create this experience, your brain must gracefully coordinate billions of signals from your eyes, muscles and skin to ensure that the hand you see, the hand you move, and the hand you touch, all feel like one thing—rather than three.
Tej Tadi, CEO and founder of Swiss neuroscience start-up MindMaze, is on a mission to reverse engineer how our brain integrates all these different signals into the unified experience of what it’s like to be you.
YOU MAY ALSO LIKE: Automating Rehabilitation With Robots for Faster Recovery
“Timing is so important to our sense of body ownership and agency,” he explains. “If there’s too much of a lag between sending the signal to move your hand, and you seeing it move, then you quickly start to lose the sense of this being your hand that you are controlling.”
While studying for a master’s degree in virtual reality (VR) and computer graphics at The Swiss Federal Institute of Technology (EPFL), Tadi spent time in a local hospital in order to research his thesis on simulating human locomotion. It was here that he saw first hand how drastic consequences can be when this brain-body synchronization breaks down. Examples included patients with schizophrenia who felt that their body was being controlled by someone else, anorexia sufferers whose perception of their body was utterly at odds with reality, and amputees who couldn’t shake the sensation that their amputated limb was still present despite clear visual evidence to the contrary.
“This gave me a lot of insight into the suffering of a wide range of populations,” he says. “But I quickly realized that the larger healthcare companies just weren’t incentivized to do anything about it.”
The Neuroscience of Body Swaps
With a background in electrical engineering and special effects design, Tadi was perhaps not expected to develop a new treatment for neurological disorders. His solution was equally as unexpected, involving neither surgical nor pharmaceutical intervention, just a motion capture camera and a 27-inch computer monitor.
After all, he thought, if such disruptive neurological conditions could be induced by something as simple as a synchronization failure, then perhaps the solution could be equally simple.
So in 2006, Tadi moved from computer graphics to neuroscience to begin a PhD in the lab of EPFL Professor Olaf Blanke, who specialized in exactly the sort of disorders of brain-body synchronization that had captured Tadi’s attention.
“My focus was about looking into how you could tweak a virtual stimulus to evoke a different response and thereby trick the neural mechanisms that modulate different aspects of the brain,” Tadi explains. ” So, if we showed you a virtual hand, we wanted to look at how we might change this in order to get you to respond as if it was your real hand.”
Tricking people into mentally incorporating external objects into their sense of bodily self is, it turns out, is surprisingly easy to achieve. In 1998, cognitive scientists Matthew Botvinicka and Jonathan Cohen showed that when participants observed a rubber hand being stroked, while simultaneously feeling their own hand being stroked, a significant proportion began to experience the rubber hand as if it was their own.
The “rubber hand illusion” has since become something of a cottage industry in cognitive neuroscience. Over the past 22 years, hundreds of studies have explored a range of variations to the individual paradigm: whether it can be induced through making a VR hand pulse in synch with the participant’s heartbeat; whether it alters blood flow to the real hand; whether it manifests differently in expert pianists, and so forth.
Tadi’s goal was somewhat more ambitious: He wanted to see if the illusion could be extended to the entire body. Together with fellow graduate student Bigna Lenggenhager, he placed participants into a VR simulation where they observed an avatar having its back stroked in synchronization with the stroking of their own body. All experienced a drift in their sense of location, away from their own physical body and into the avatar itself.
Virtual Therapy
Just as virtual experiences could cause people to connect with external objects, Tadi believed the same approach could be used to repair a lost connection with one’s own body. So, as he came to the end of his PhD in 2013, he founded MindMaze, and in 2015 the company launched its first product, a neurorehabilitation platform called MindMotion, which uses motion capture to record a person’s movements and recreate them in a pair of virtual hands.
Their first application was for the rehabilitation of stroke patients. A stroke occurs when a loss of blood supply to the brain causes neurons to die from the lack of oxygen, tearing apart the intricate neural wiring responsible for coordinated movement and leaving many patients with partial paralysis.
Your brain’s wiring diagram is not a static thing, however, but something developed throughout childhood. When neurons—such as those involved in trying to move your arm and sensing that it has moved—regularly fire together, the connections between them strengthen. This process, called neuroplasticity, underpins how we learn, whether that’s the association of words and objects, or the association of muscle impulses and movements.
Post-stroke therapy aims to help patients redevelop this wiring diagram, through the repeated attempt to complete basic object manipulation tasks, such as moving, squeezing, and rotating a rubber ball. Success depends on how frequently the patient engages in these often highly frustrating tasks—especially in the first few weeks post-stroke, when the destroyed neural wiring is at its most pliable.
“With stroke patients, the earlier you start [post-stroke therapy], the better the chances for it to work out, so we want to help capitalize on that window of recovery.”
—Tej Tadi, CEO and founder, MindMaze
“The rationale for the MindMotion platform is about accelerating neuroplasticity,” Tadi explains. “With stroke patients, the earlier you start, the better the chances for it to work out, so we want to help capitalize on that window of recovery. ”
By situating the same movement tasks in the context of a range of motion capture-controlled computer games, from racing a car to flying a plane, the MindMotion platform aims to increase the level of engagement, while allowing participants to run through their recovery regime without the constant attention of a dedicated physiotherapist. Though the feedback is virtual, as in the rubber hand illusion, the mere synchronization between the attempted movement and the visual feedback is enough to create a convincing feeling of real-world success, triggering the restrengthening of the neural pathways involved.
Creating a Brain-Inspired Chip
MindMotion is now being developed for the treatment of other neurological disorders, such as Parkinson’s and multiple sclerosis. But putting neural networks back together again is only half of Tadi’s ambition for MindMaze.
The other is to use what the company has learned about how the well-functioning brain processes multiple information streams in parallel to create a brain-inspired computing platform designed at the hardware-level to do the same.
After all, the task of a neuroscientist trying to make sense of a person from many noisy streams of neural, physiological, and behavioral data is very similar to that of a brain trying to make sense of a world from sensory inputs. And it’s a task for which traditional computer chips—designed to serially process a chain of step-by-step commands—are strikingly poorly suited.
MindMaze’s solution is a brain-inspired computing platform, called the CogniChip, currently under development based partly on hardware already created for the MindMotion platform.
“The CogniChip will give us the ability to compute and combine a range of biosensing data, whether that’s muscle activation, brain signals, motion capture, or finger-tracking,” Tadi says. “And, unlike general-purpose computer chops, it will allow us to do so with very low power requirements. ”
Elements of the CogniChip, Tadi says, are already being utilized as part of a partnership with British racing team McLaren Racing to analyze neural data from Formula One drivers.
While this is initially focused on detecting damage to brain function in the case of an accident, the longer-term aim is to integrate this data with telemetry gathered from the extensive array of sensors distributed throughout McLaren’s Formula One cars in order to gather live insights into driver performance.
“The [Formula One] driver is making millisecond judgments in a very extreme environment, so any insight we can get into how they’re responding throughout a race could really make a difference.”
—Tom Dix, engineer, McLaren
“Formula One is very mentally driven,” explains McLaren engineer Tom Dix, who has been helping test the headset outside of race weekends. “The driver is making millisecond judgments in a very extreme environment, so any insight we can get into how they’re responding throughout a race could really make a difference.”
That intense environment also makes it challenging for MindMaze’s specially designed headset to extract the tiny electrical signals emitted through the skull when populations of neurons fire—the basis of a neuro-measurement technique called electroencephalography. The main result of initial testing has been able to show that their platform is able to consistently extract these signals despite the electrical noise and physical forces generated by a race car traveling at 200 mph.
That challenge may make Formula One seem like a strange place to develop this platform given, as Tadi points out, the clear applications for detecting mental states like fatigue in more sedate modes of transportation.
For Tadi’s approach to technology development, however, it’s precisely this challenge that makes Formula One ideal.
“There’s an obvious application into aviation or long-haul truck driving,” he says. “But our approach has always been to prove things at the highest level of difficulty before translating them into a consumer environment. That’s what we’ve done with by following the medical route with MindMotion, and that’s what we’re doing in Formula One.”