NHK working to improve TV sign language

Friday 02 Dec 2011
Researchers in Japan are working to improve sign language for television viewers by creating computer generated presenters who use facial expressions as well as hand signs.

The research is being carried out by staff of the NHK Science & Technology Research Laboratories in Tokyo, run by Japan’s public broadcaster.

NHK broadcasts news and other programmes in Japanese sign language (JSL) for the hearing impaired and says it has received requests for more such programmes.

Researcher Hiroyuki Kaneko reported on the work in the latest issue of the laboratories’ newsletter, Broadcast Technology.

The researchers have captured the facial expressions of a JSL user, including the movement of the lips, jaws, cheeks and eyebrows, and reproduced them on a computer generated (CG) face.

“In future work, we will create CG animation generation technology to produce JSL sentences combining both manual and non-manual signs.”

Mr Kaneko said JSL places importance on facial expressions and the shape of the mouth in addition to hand signals.
 He gave the example of a commonly used JSL sign that involves waving a paper fan beside the face. This can mean ‘summer’, ‘hot’, ‘south’ or ‘paper fan’, depending on the facial expressions that accompany it.