An Overview of Sign Language Detection Using Wearable System

Abstract

This paper presents a survey of sign language detection using a wearable system by using various techniques. This technology is used for deaf and dumb as they face many difficulties in their day to day life to communicate with the world. As many people who are not deaf and dumb unable to understand the sign language. To make this possible these gloves are made so that a deaf and dumb people can communicate with normal people. The parameter used are flex sensors, Hand gloves, Bluetooth Module, Voice synthesizer, Capacitive charge sensor, Microcontroller, Gyros, Lillypad. This device can store many unique gestures and translate them into text or speech. Thus the device will be a low-cost device and can be benefitted to many peoples. A lot of research work is done to improve efficiency and accuracy.

Introduction

It was always a serious problem for deaf and dumb people to communicate with each other but somehow they came up with sign language as a solution. Deaf and dumb people face problem in communication to normal person. As For communicating to each other, both need to have knowledge about sign language. Which is bit difficult scenario. Generally normal person can communicate with deaf and dumb people but as they are not aware of all the signs and terminologies it is difficult for them to communicate.

Normal speed of any deaf and dumb person of talking is more than normal person even after a normal person learns sign language. There must be an intermediate system Hence proposed system will be a hand gloves based sign language to interpreter in following format.

• Sign to character interpretation. (e.g. ‘A’, ‘B’, ‘C’)
• Sign to word interpretation. (e.g. ‘Hello’, ‘Hi’, ‘I’, ‘Me’)
• Sign -> Word -> Speech.

Literature Survey

According to the Word Federation of the Deaf, there are around 72 million mute, deaf or deaf mute in the world. Many of them communicate with each other with the help of sign language. But many people who are not deaf & mute cannot understand sign language.

Thus a glove is made so that deaf and mute people can communicate with the normal people. This glove convert the sign language into text. A LCD display is attached to the glove as shown in Fig1 when a person use sign language it convert sign into text which will be displayed on LCD so that the person can understand the sign. Some signs or gestures are stored in the database to identify the sign language. It consist of six flex sensors for sensing the movement of hand or finger bends and a microcontroller circuit which transmit signal to a Mini-LCD in order to display the resulting letter. Microcontroller is the heart of system

There are few limitation in the using of this hand glove as follows:

1. Due to use of 4 inch flex sensor accuracy is average.
2. We can store only limited sign language.
3. If someone is blind he can’t communicate with it because it converts sign into text.
4. Sign language is converted into Text not in Speech.

The different Gloves can be used according to the user’s needs: 

• Sayre Glove: Light band sensors with flexible tube with a light source at one end and photocell at other.
• Digital Data entry glove: Touch and proximity sensor for sensing thumb touching another part of hands, wrist bend sensor for fingers and inertial sensors at wrist to sense twisting of hands.
• Data Glove: 5 to 15 sensors made of flexible tubes with a detector at one end to detect direct and reflected rays and a light source at other.
• Power Gloves: Resistive sensor are used for overall flexion measurement of thumbs and fingers.
• Space Gloves: Sensor with 12 bit ADCs that measure the flexion of metaphalangeal join.
• Pinch Glove: Has electrical contact on the finger and the thumb. Different combinations are made by completing a conductive path when two or more electrical contact meet.

Limitation:

1. The System is not portable.
2. The system is not real-time.
3. Accuracy can be improved by connecting more sensors.

In this paper, the authors have presented a wireless hand gesture recognition glove for the Real-time translation of Taiwanese sign language. Sign language was the most expressive way for the deaf and dumb people to connect to the outside world and make communication way easy.[3]

The authors have designed the most comfortable and impressive system for these people. They have divided the system architecture in two different parts

1. Inertia sensors based system
2. Video-based system.

In Inertia based system, the system consists of the glove, mobile device, flex sensors and inertia sensors. The gesture recognizing glove and mobile device both are connected to each other through Bluetooth. The flex sensors are used to detect which fingers joints are bent. As they could not find it sufficient using flex sensors it was not giving the proper accuracy. They also used inertia sensors for more gestures. The inertia sensors and the flex sensors are used as sensing components of the System to generate the input signals. The periodically generated signals are stored in the queue first, then the stored signals will be processed by the algorithm to calculate the repeated times of hand gestures in sequence.

In video-based system, hand tracking as well as 3D motion trajectory is used. It also consists of hand tracking camera and 3D motion trajectory database. Firstly the 3D motion is detected with the sign language in database and then compared with hand tracking camera and then the results are displayed on the screen.

Disadvantages

1. In the video-based system it is difficult for the complex background.
2. It is inconvenient for the deaf and dump people.
3. There is also an issue of light condition.

Limitation: In video system there is problem of accuracy.

Hand talk Implementation of a gesture recognizing Glove is developed for creating artificial voice with the help of following gestures. For creating artificial voice all the devices are setup on hand gloves. This Gloves was created by electronic visualization lab. This Gloves was first commercially used by Nintendo Entertainment system named after Nintendo Power Glove in year 1987. [4]

For creating artificial voice the devices like sensors are fitted or setup on the glove. The sensors used are:

1.  Flex sensors.
2. Bluetooth module
3.  Microcontroller

This Technology is also able to character recognition along able to character recognition along with a learning algorithms to learn new symbols and gestures. These gloves can also be fitted with extra sensors like gyros and accelerometer for calculating the position of hands and palm and able to make a new gestures. As one gesture can have one or more meaning. So accelerometer can be used.

The proposed solution was the glove with properly fitted design of sensors to generate artificial voice. The following proposed system works on

1. Hardware Module
2. Software Module

Hardware Modules are:

1. Flex sensors.
2. Bluetooth module
3. Microcontroller
4. Gyros

Software Module are:

1. Microcontroller Software
2. Mobile Software
3. Text to Speech Synthesizer

Advantage:

•  Low-cost device.

Limitation:

1. Complex Design
2. Complex Algorithm

Sign language detection devices are available and typically limited in portability, it also requires a sophisticated and heavy computation make these devices expensive and bulky unsuitable of in-field application.[5]

In this paper they present a glove which is based on the charge transfer touch sensors which translate the American Sign Language into the words, the translation is based on the charged touch sensor which gives a binary detection system which gives a set of digital touch sensor instead of an analog signal.

The problem with the analog signal is that it requires more complex computation and has less accuracy than the digital system.

In this Glove based technology the capacitive touch sensors are mounted on the fingertips of each finger and they are selectively activated which causes a wide range of selection, the combination of sequence is generated from the glove which gives output as a character which can be further combined by the processing unit to form the words and those words are form sentences. The Processing unit processes input by using various type of algorithm to process the data.

This devices assign specific binary values of each character which can be used for recognition of all 26 English alphabets and 10 numeric digits signed in the American sign language (ASL).

Limitation:

1. Complicated in design.
2.  Line of code is also complicated, it requires a more lines of code than the other sign language detection devices.

Conclusion

In this review paper, different techniques of sign language recognition are reviewed on the basis of sign acquiring methods and sign identification methods. For sign acquiring methods, vision based methods and for sign identification methods, artificial neuron network proves a strong candidature.

References

1. Hanine Elhayek, Jessica Nacouzi, Abdallah Kaseem, Mustafa Hamad, Sami El-Murr “Sign To Letter Translator System Using Hand Glove”, International IEEE @2014
2. Dheeraj gupta, pankhuri Singh, Khushbu Pandey, Jaya Solankhi ”Design And Development Of A Low Cost Electronic Hand Glove For Deaf And Blind”, IEEE@2015
3. Lih-jen Kau, Wan-Lin Su, Pei-ju yu, Sin-Jhan Wei “Real Time Portable Sign Translation System ”
4. Celestine Preetham, Girish Ramakrishnan, Sujan Kumar, Anish Tamse, Dr. Nagendra Krishnapura, ”Hand Talk Implimentation Of A Gesture Regognizing Glove”, IEEE@2013
5. Kalpattu S. Abhishek, Lee chumn Fai Qubely, Derek Ho , “Glove Based Hand Gesture Recognition Sign Language Translator Using Capacitive Touch Sensor ” IEEE@2016