A recent article in Science Advances touted the creation of a mind-controlled hearing aid that allows the wearer to focus on voices. The device was by scientists in a laboratory at Columbia University in New York. The scientists, led by Nima Mesgarani, believe the device could transform the ability of persons with hearing loss to communicate more effectively in noisy listening situations.
The key to the device is that it mimics the brain’s natural ability to single out and amplify one voice against background conversation. Unsurprisingly, even today’s most advanced hearing aids do not separate specific voices from the cacophony of sound experienced by many hearing aid wearers in crowded environments.
In the Science Advances article, Mesgarani said, “The brain area that processes sound is extraordinarily sensitive and powerful. It can amplify one voice over others, seemingly effortlessly, while today’s hearing aids still pale in comparison.”
Using AI, Brain Sensors, to Focus on Specific Voices
Known by many as the the cocktail party effect, scientists have worked for decades to improve a hearing aid wearer’s ability to understanding specific voice in the presence of other similar voices. The brain-controlled hearing aid appears may have cracked this problem using a combination of artificial intelligence and sensors designed to monitor the listener’s brain activity.
The hearing aid first uses an algorithm to automatically separate the voices of multiple speakers. It then compares these audio tracks to the brain activity of the listener. Previous work by Mesgarani’s lab found that it is possible to identify which person someone is paying attention to, as the person’s brain activity tracks the sound waves of that voice most closely.
The device compares the audio of each speaker to the brain waves of the person wearing the hearing aid. The speaker whose voice pattern most closely matches the listener’s brain waves is amplified over the others, allowing them to effortlessly tune in to that person.
The scientists developed an earlier version of the system in 2017 that, while promising, had the major limitation that it had to be pre-trained to recognize speakers’ voices. Crucially, the latest device works for voices it has never heard before. To test the device, the lab recruited epilepsy patients who already had electrodes implanted in their brain to monitor seizure activity ahead of planned brain surgery. The patients were played audio of different speakers simultaneously while their brain waves were monitored via the electrodes implanted into their brain.
An algorithm tracked the patients’ attention as they listened to different speakers that they had not previously heard. When a patient focused on one speaker, the system automatically amplified that voice, with a lag of just a few seconds. When their attention shifted to a different speaker, the volume levels changed to reflect that shift.
Mind Controlled Hearing Aids Available in the Next 5 Years?
The current version of the hearing aid, which involved direct implants into the brain, is unsuitable for commercial use at this time, however the researchers believe it will be possible to create a non-invasive version of the device within the next five years, which would monitor brain activity using electrodes placed inside the ear, or under the skin of the scalp.
In theory, Mesgarani said, the device could also be used like a pair of audio “binoculars” to covertly listen in on people’s conversations, although this was not the intended application. The next step will be testing the technology in those with hearing impairments.
One question is whether it will be as easy to match up brain activity in people who are partially deaf with sound waves from speech.
According to Jesal Vishnuram, technology manager at the charity Action on Hearing Loss, said that one of the reasons people find conventional hearing aids unpleasant in noisy environments is that their brain is out of practice at filtering sounds and so does this less effectively. “One of the reasons people struggle is that they often wait a long time before getting a hearing aid and in that time the brain forgets how to filter out the noise and focus on the speech,” she said. “This is really interesting research and I’d love to see the real world impacts of it.”