Most of us instinctively watch speakers' lips and gestures in crowded restaurants or busy streets to better understand what they're saying. Now, groundbreaking research at the University of Rochester aims to unravel how our brains merge these visual cues with sound to enhance speech comprehension in noisy environments.
Why it matters
Understanding this complex brain process could revolutionize technology for the deaf and hard of hearing, particularly those who use cochlear implants. This research could bridge gaps in understanding and supporting speech comprehension.
The challenge
Studying how the brain processes speech is a complex puzzle. The visual cortex (at the back of the brain) and auditory cortex (on the temporal lobes) must work together, but researchers still don't fully understand how this integration happens.
An electroencephalography (EEG) cap, which measures the electrical activity of the brain through the scalp, collects a mixture of signals coming from many different sources.
The approach:
- Researchers will monitor the brainwaves of 250 cochlear implant users while they watch and listen to speech
- The study will compare people who received implants at different ages.
- Scientists hypothesize that those who got implants later (around age 12) may rely more heavily on visual cues than those implanted earlier (around age 1).
- The team uses EEG caps to measure brain activity during speech processing
Backstory
The team's previous research has already yielded insights into how specific mouth movements help listeners distinguish between similar-sounding consonants like "F" and "S" or "P" and "D" in noisy situations. This new phase takes this a step further by examining more natural and continuous speech patterns.
The intrigue
The research team faces unique technical hurdles that require innovative solutions. EEG caps collect mixed signals from various brain sources, and cochlear implants generate electrical activity that interferes with readings. To address these challenges, the University of Rochester will employ advanced signal processing and computational modeling.
We'll keep you posted on future developments.
