Author: Stephen Johnson / Source: Big Think
- The technique involves training neural networks to associate patterns of brain activity with human speech.
- Several research teams have managed to get neural networks to “speak” intelligible words.
- Although similar technology might someday help disabled people regain the power to speak, decoding imagined speech is still far off.
Several research groups have recently made significant progress in using neural networks to convert brain activity into intelligible computer-generated speech, developments that could mark some of the first steps toward radically improving the quality of life for people who’ve lost the ability to speak.
As a recent article from Science notes, the groups, which have published several separate papers on the preprint server bioRxiv, aren’t yet able to convert people’s purely imagined words and sentences into computer-generated speech. Still, the teams were successful in getting neural networks to reconstruct words that various participants had either heard, spoken aloud or mouthed silently.
To accomplish that, the teams recorded brain signals and fed them to a neural network, which then matched the signals with associated sounds or mouth movements.
Unfortunately, this kind of work requires opening the skull; researchers need extremely precise data that can only be obtained by surgically implanting electrodes directly onto regions of the brain associated with speech, listening or motor functioning. Making matters more complicated is the fact that each person shows unique neural activity in these regions, so what an AI learns from one person doesn’t translate to the next.
“We are trying to work out the pattern of … neurons that turn on and off at different time points, and infer the speech sound,” Nima Mesgarani, a computer scientist at Columbia University, told Science. “The mapping from one to the other is not very straightforward.”
For the research, the teams relied on participants who were already scheduled to undergo invasive surgery to remove brain tumors or receive pre-surgery treatments for epilepsy.
One team, led by Mesgarani, fed a neural network with data from participants’ auditory cortexes that was obtained while they listened to recordings of people telling stories and listing numbers. Using the brain data alone, the…
The post How AI is learning to convert brain signals into speech appeared first on FeedBox.