Lero AR

AR project brings sign language into lecture theatres

Prof Brian Fitzgerald, Elayne Ruane, Dr Anthony Ventresque, and Thomas Laurent

5 July 2018

Researchers from software research centre Lero and University College Dublin have begun a research initiative with Microsoft designed to enable future communications between deaf and non-deaf people.

The project, which will develop a virtual Irish sign language interpreter, is backed by SFI in collaboration with Microsoft. It incorporates various Microsoft technologies that enable interaction without the need for a human interpreter or sign language knowledge by the non-deaf individual.

“This is an exciting research programme which could transform the ability of deaf people in the future to communicate across education, work, social and family situations with non-deaf individuals and groups who do not know sign language,” said Dr Anthony Ventresque, Lero researcher and director of the UCD Complex Software Lab who is leading the programme with PhD students Elayne Ruane and Thomas Laurent.


“Our business is all about communications which is why we began to think that deafness should not be a barrier,” commented Ross Smith, director of engineering at the Microsoft Skype Division. “The goal of the three to five year programme is to develop a solution that could be made available to deaf people worldwide as part of our Skype4Good global initiative.”

Lero researchers at UCD have already developed a prototype using Microsoft tools including HoloLens mixed reality smartglasses, LUIS.AI language understanding, Azure cognitive services and Xbox depth camera technologies. An avatar translates the sign language of the deaf person into speech and vice versa.

Dr Ventresque said: “To date we’ve developed a sign language interpreter that focuses on hand gestures; the next step is to make the prototype realistic and useful to the end user by integrating facial expression recognition and generation into the interpreter, which is a critical feature missing from current offerings in this area. We feel we also have a head start in this space as we’ve developed an innovative solution to emotion detection in the context of another joint Lero Skype project.”

Leah Ennis McLoughlin, a deaf UCD science student, said: “We’re all part of the same world so it would be fantastic if this smashes linguistic barriers which currently exist for deaf people across work and social interaction.”

TechCentral Reporters

Read More:

Comments are closed.

Back to Top ↑