Beskrivning av bidrag

In Sweden alone, there are around  8,000 to 10,000 people who are born deaf or severely hearing impaired, and around 30,000 people who use sign language. In these people’s lives are families, friends, partners, and colleagues who must learn sign language in order to ensure reliable communication ( These statistics demonstrate a population of individuals in need of sign language tutoring. However, teacher-led tutoring is not always available or affordable those other than the hearing-impaired themselves. Additionally, watching videos or viewing photo tutorials to learn is difficult for many as they cannot directly interact, ask for the signs to be shown again, do sign language tests, and so on. This can create barriers for individuals whose goal is to simply acquire skills of sign language – communicate with their hearing-impaired friend becomes impossible.

This is where our project come in.

Using the Nao robot programmed with the official software “Nao Software” we were able to create this project giving the various input methods that comes with the robot, to help both the hearing impaired and their loved ones. We hope to ensure a better quality of communication and life to those afflicted with hearing disabilities and others in their communities.

By programming the robot into two different modifications – learn and practice – users have the opportunity to develop and hone their sign language skills. The robot demonstrates a preferred sign requested by the user utilizing oral input and voice recognition, and practice; the robot performs a sign that the user guesses, again, via voice recognition.

Our vision and hope as researchers is that this project will lead to others exploring the possibility of using social robotics for aiding or replacing traditional sign language tutoring resources and, perhaps, developing social robots specifically for the purpose of sign language tutoring.


Vincent Wong
Malin Augustsson
Fredd Duque Farias
I Wayan Kurniawan Aditya Wardana


Interactive Media Technology, KTH