Make it possible to write to someone who can no longer move.
The ambition is worthy of a sci-fi movie, but a team of researchers from Stanford University (California) has taken a big step in this direction.
Thanks to an approach based on the analysis of brain activity and artificial intelligence, they were able to make a first paralyzed patient express himself in thought, twice as fast as with all existing communication devices.
Until now, previous techniques were content to interpret eye movements to deduce instructions.
Stanford researchers approached the question from a new angle: even when the execution of gestures is no longer possible, the neural circuits that control them are still functional.
It would therefore suffice to let the disabled person imagine writing, and decode the activity of his brain to transcribe it in full.
Putting together letters from neural signals
The first tests were carried out on a 65-year-old participant, whose damaged spinal cord no longer allows him the slightest movement below the neck. Two sensors were implanted on his skull. "They measure 4 mm by 4 in area, each equipped with 100 electrodes thin as hair," describes Jaimie Henderson, professor of neurosurgery who participated in the experiment. These sensors are inserted about a millimeter into the outer surface of the brain, through routine neurosurgery. "
When the patient imagines himself writing the letters one by one, the electrodes detect the electrical activity in his neural circuits, different depending on the letter.
An artificial intelligence then reconstructs the characters from the electrical signals, among the 26 letters of the alphabet and the 5 punctuation marks that it is able to identify.
And even by thought, the signs are not always written perfectly: it is easy to confuse an "h" with an "n", or a "u" with a "v".
An automatic corrector similar to that of your smartphone then corrects any mental typos.
The interface can recognize 26 letters and 5 punctuation marks in handwriting.
F. Willet et al.
/ Nature 2021 $
The result is astounding: the participant himself was "pleasantly surprised that the system could interpret what he was thinking," recalls Professor Henderson.
“He often had the impression of visualizing the feet of a fly, which we could not transcribe, but it worked all the same!
"
Much faster in handwriting
In fact, this new form of brain-computer interface (
brain-computer interface
, or BCI, English) allowed his first user to speak at a speed of 90 characters per minute, against 47.5 at the for pre-existing devices. The patient thus came considerably closer to the 115 characters per minute that can be written, on average, by an adult of his age without any disability. The character recognition algorithm also shows high performance, identifying the correct letter in over 94% of cases, and over 99% after using the automatic corrector.
For Jaimie Henderson, these performances can be explained by the precision of the handwriting, where the pre-existing interfaces relied only on rectilinear movements, in particular with the eyes. "The patterns to be recognized were only a series of straight lines which differed only in terms of angle and length," explains the researcher. But handwriting is curvilinear and more complex, it makes it possible to obtain signals at the level of neurons that are easier to distinguish between them, the probability of confusing them is lower ”, which also makes it possible to write much faster.
Now that their proof of concept is established, the Stanford researchers are considering replicating their device for other patients, reducing the time it takes to train their artificial intelligence to recognize the letters. Jaimie Henderson even plans to take an interest in "a similar device that would decode attempts at speech-related movements, and translate them, orally or into text." »Understanding, an even greater promise: to give a voice to silent people.
