Sign Language Translation Devices

Occasionally, a signing deaf person comes to the clinic seeking services and while not totally lost, it is always a challenge for most of us.  The deaf are so very kind to us and want to learn what we must say, but it is difficult to get our messages across. The problem is not a lack of professionalism in working with the deaf population, it is the lack of practice with sign language. Clinical audiologists can usually sign, but most of us are grossly out of practice as we do not use sign language on a routine basis.  Without much practice, it is still possible to get the point across, but it when the patient answers that is where we get into trouble.  The signs come so fast that it is difficult to follow them and even if your vocabulary is up to speed, visual perception is not.  The communication breakdown comes with the interaction, not the one-way presentation of information. 

What about a smart device that could translate sign language while being worn on the wrist of the signer….that could be a bridge to communication between the signing deaf and those who don’t know sign language (or those that forgot the signs and/or cannot perceive them properly).

Harrington (2016) indicates that American Sign Language (ASL) is the fourth most-used language in the United States and possibly the third most-used non-English language in the U.S. This claim has been around since the early 1970s. The strongest claim for these numbers come from Mitchell, Young, Bachleda, and Karchmer (2006). These researchers at the Gallaudet Research Institute, traced the origin of this claim to a single study done in 1974. That study, the National Census of the Deaf Population (NCDP), concluded that the total number of sign language users in the U.S. was close to a half-million strong.  These days, studies estimate that the number that the real number of ASL speakers is from 500,000 to two million in the U.S. making it the leading minority language, right after the “big four”: Spanish, Italian, German, and French. 

  Researchers at Texas A&M University have developed a wearable device that could eventually help transmit clinical information to those that use sign language.  Their device “translates” sign language into English by sensing the user’s movements.  Dr. Roozbeh Jafari, at the Texas A/M, Department of Biomedical Engineering, within the Center for Remote Health Technologies and Systems led the development project. Dr Jafri’s research interest lies in the area of wearable computer design and signal processing.  The device for use to translate sign language is still in the prototype stage but can already recognize some 40 ASL signs with 96 percent accuracy.

It works by the use of two very specific sensors. The first is a motion sensor that uses an accelerometer and a gyroscope to measure the signer’s hand and arm – speed and angle. By sensing where a user’s hands and arms are, the device can begin to guess what word signer might be signing. There is also a electromyographic sensor, that measures the electrical potential of muscle movement. This sensor is critical to the function of the device as it can tell exactly which part of the hands and fingers are moving.  Dr. Jafari feels that, “If you look at the American Sign Language vocabulary, there are cases where the hand itself is moving and then you have very fine-grained movement of the fingers, if you want to detect those, you’re not going to be able to use just the motion of the hand.” As someone who has long worked with wearable technologies, such as watches that monitor heart rhythm, Jafari understands the importance of comfort and aesthetics. If a device is uncomfortable and obtrusive, people won’t wear it. The current prototype of the sign language translation device looks like a medical implement, with electrodes and straps and wires. As the technology develops into a usable form he feels that it should be small and attractive. 

A Glove Project among Glove Projects

Another project that seems to have come along well is that of two undergraduate students at the University of Washington,  Navid Azodi and Thomas Pryor.  They split the Lemelson-MIT prize for the most innovative students by their invention of SignAloud, a device that also translates ASL into speech or text.  Their idea initially came their belief that communication is a fundamental right. 

According to a post at the American Academy of Audiology (2016), their invention, SignAloud, is “…a pair of gloves that can recognize hand gestures that correspond to words and phrases in American Sign Language. Each glove contains sensors that record hand position and movement and send data wirelessly via Bluetooth to a laptop computer. The computer looks at the gesture data through various sequential statistical regressions, similar to a neural network. If the data match a gesture, then the associated word or phrase is spoken through a speaker.” Click on their picture to see how the system actually works 

There seems to be continuing interest among engineering students in the translation of sign language, some use a glove and other input techniques check out You Tube for some other interesting and creative projects.  Any of these technologies, once developed, would make life much better for the 500,000 to 2 million US and the estimated 70 million hearing impaired sign language users worldwide…….  No matter if its gloves or watches this is the technology that could revolutionize the life of the manually deaf and their interactions with the hearing community.    Dr. Jafari’s new type of watch, however, would also reduce the anxiety of audiologists that have not used their sign language skills for quite some time and be a bit more convenient than gloves.

References:

American Academy of Audiology (2016).  SignAloud Invention.  Retrieved February 20, 2017.

Callis, L. (2014).  2014:  Deaf culture totally had a moment.  Interpreting Services, retrieved February 20, 2017.

Harrington, T. (2016). Sign Language:  Ranking and number of users.  Gallaudet University Library. Retrieved February 20, 2017.

Mitchell, R., Young, T., Bachleda, B., & Karchmer, M. (2006).  How many people use ASL in the United States?  Why estimates need updating.  Sign Language Studies, Volume 6, Number 3, 2006,

     Retrieved February 20, 2017.

Wanshel, E. (2016).  Students invent gloves that can translate sign language into speech and text.  Huffington Post.  Retrieved February 20, 2017.