:quality(70)/cloudfront-eu-central-1.images.arcpublishing.com/irishtimes/Y6MNQU3J4YOMMPZFWXJHX76R3M.jpg)
A brain decoder that allows people literally to “speak their minds” could one day help doctors communicate with patients who cannot talk.
The technology generates synthesised spoken words from brain signals associated with movements of the jaw, larynx, lips and tongue, allowing scientists to tap into the verbal thoughts of volunteers
All the volunteers, four women and one man, in the study were epilepsy patients who had undergone surgery to implant an electrode array on to their brain surfaces.
At the start of the study, patterns of electrical activity were recorded from the brains of the five volunteers as they spoke several hundred sentences aloud.
:quality(70)/cloudfront-eu-central-1.images.arcpublishing.com/irishtimes/DBZVDZKWBFEVYCQUKFI2YKFV6U.jpg)
:quality(70)/cloudfront-eu-central-1.images.arcpublishing.com/irishtimes/T5CPXYV25HXOVLOQYSZZQ4QHDY.jpg)
:quality(70)/cloudfront-eu-central-1.images.arcpublishing.com/irishtimes/RXTF6DTZOZG6GIYTJAMC7PRPEY.jpg)
:quality(70)/cloudfront-eu-central-1.images.arcpublishing.com/irishtimes/IXWJBCKRXHXIA2GJ42HS2MCJDU.jpg)
The sentences were taken from well-known children’s stories, including Sleeping Beauty, The Frog Prince, and Alice In Wonderland.
Armed with the recordings, the US team devised a system capable of translating brain signals responsible for individual movements of the vocal tract.
Finally, the decoded signals were fed to a neural network computer linked to a voice synthesiser.
In trials, volunteer listeners were easily able to understand and transcribe the resulting synthesised speech.
The research, led by Dr Edward Chang from the University of California, San Francisco, US, is reported in the latest issue of Nature journal.
The scientists wrote: “Listeners were able to transcribe synthesised speech well.
“Of the 101 synthesised trials, at least one listener was able to provide a perfect transcription for 82 sentences with a 25-word pool and 60 sentences with a 50-word pool.”
They added: “Our results may be an important next step in realising speech restoration for patients with paralysis.”
Natural speech production involves more than 100 facial muscles, according to the scientists.
Mime sentences
In a second part of the study, one participant was asked to speak sentences and then mime saying them without making a sound.
The decoder was able to read the brain signals associated with the mime and translate them into synthesised speech.
Two experts writing in an accompanying article in Nature pointed out that speaking might seem effortless but was in fact “one of the most complex actions that we perform”.
The experts, Chethan Pandarinath and Yahia H Ali, from Emory University in Atlanta, US, added: "With continued progress, we can hope that individuals with speech impairments will regain the ability to freely speak their minds and reconnect with the world around them." – PA