By YUKI SATO/ Staff Writer
February 8, 2025 at 07:00 JST
A deaf athlete demonstrates how to use SureTalk, a sign language translator app, during an event held in Chiba on Oct. 15, 2024. (Ayako Fujita)
Telecom giant SoftBank Corp. is working with the University of Electro-Communications in Tokyo to fine-tune an app called SureTalk that converts sign language gestures into text.
When a person uses sign language in front of a camera, the AI-based app recognizes skeletal movements to choose words from its database.
It adds postpositional particles and other elements as necessary to display naturally translated text.
Words spoken by people without hearing disabilities appear as text.
“Sign language movements can be big or small depending on the speaker. So, it is important to feed (the AI) many patterns,” said Shusei Komatsu, 33, a SoftBank employee in charge of developing the app.
Komatsu has been hearing-impaired since birth himself. Registering each word requires data from 100 or so people, he said.
General users can also register sign language videos via a website and an iPhone app.
Initial research started in October 2017.
SureTalk has been available to municipal governments on a trial basis since November 2020.
The app currently has about 5,000 words in its database and is used by around 20 local governments, including the Tottori and Fukuoka prefectural governments.
It has been used at events for the Tokyo Deaflympics, an international sporting event for hearing-impaired athletes that will be held for the first time in Japan in November, and on other occasions.
“Sign language is the first language for the hearing impaired. They turn Japanese text into sign language to understand,” Komatsu said. “I hope that what people without hearing problems say will be directly converted into sign language one day.”
He also voiced hopes that the app will feature at the counters of airports, hotels, hospitals and other facilities.
A peek through the music industry’s curtain at the producers who harnessed social media to help their idols go global.
A series based on diplomatic documents declassified by Japan’s Foreign Ministry
Here is a collection of first-hand accounts by “hibakusha” atomic bomb survivors.
Cooking experts, chefs and others involved in the field of food introduce their special recipes intertwined with their paths in life.
A series about Japanese-Americans and their memories of World War II
