For millions of deaf and hard of hearing, communication often presents significant challenges. Now, researchers at Florida Atlantic University have developed an AI technology that can translate American Sign Language (ASL) gestures into text in real-time, using only a standard webcam.
Why it matters
By providing real-time translation, this technology could empower approximately 11 million deaf people in the U.S. to navigate daily interactions without depending on limited interpreter resources.
How it works
The system pairs two technologies:
- YOLOv11, an object detector, identifies hand shapes.
- MediaPipe tracks 21 keypoints per hand (like fingertips and knuckles) to map gestures.
- Analysis of live video from a standard webcam, translates gestures into text with minimal delay.
Bader Alsharif, the first author, demonstrates how the system spells out names and locations in real time using American Sign Language.
By the numbers
-
98.2% accuracy: Tested across diverse lighting and backgrounds.
-
130,000 images: Training data included varied hand sizes, skin tones, and environments.
-
21 keypoints: Tracked per hand to distinguish tricky gestures (e.g., “A” vs. “T”).
The challenge
ASL tech often stumbles with:
-
Similar gestures: Letters like “M” and “N” look nearly identical.
-
Real-world chaos: Motion blur, dim lighting, or cluttered backgrounds confuse most systems.
The team tackled this by training their AI on a massive, diverse dataset, ensuring it works reliably in unpredictable settings.
The takeaway
This AI system bridges communication divides with empathy and rigor. Next, the team aims to interpret full ASL sentences, enabling fluid conversations.
Healthy hearing starts here
Learn about the health of your hearing with a free 15-minute hearing screening by an audiologist.
★ Call 708-599-9500 to schedule your free screening.
★ For facts about hearing loss and hearing aid options, grab your copy of The Hearing Loss Guide.
★ Sign up for our newsletter for the latest on Hearing aids, dementia triggered by hearing loss, pediatric speech and hearing, speech-language therapies, Parkinson's Voice therapies, and occupational-hearing conservation. We publish our newsletter eight times a year.
Don't let untreated hearing loss spoil your enjoyment of life.
Crest Hill, IL - 630-633-5060 | Palos Hills, IL - 708-599-9500
