Sign language recognition by image analysis

---

Tezin Türü: Yüksek Lisans

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Mühendislik Fakültesi, Elektrik ve Elektronik Mühendisliği Bölümü, Türkiye

Tezin Onay Tarihi: 2015

Öğrenci: BUKET BÜYÜKSARAÇ

Danışman: GÖZDE AKAR

Özet:

The Sign Language Recognition (SLR) Problem is a highly important research topic, because of its ability to increase the interaction between the people who are hearing-impaired or impediment in speech. However there are several limitations of the existing methods. Most applications need different necessities like making the user wear multi-colored or sensor based gloves or usage of a specific camera. We propose a simple but robust system that can be used without the need of any specific accessories. The proposed system consists of three main steps. First we apply segmentation to the face and hand region by using Fuzzy C-Means Clustering (FCM) and Thresholding. FCM is a clustering technique which employs fuzzy partitioning, in an iterative algorithm. After the face and hands are segmented, the feature vectors are extracted. The feature vectors are chosen among the low level features such as the bounding ellipse, bounding box, and center of mass coordinates, since they are known to be more robust to segmentation errors due to low resolution images. In total there are 23 features for each hand. After the feature vectors are extracted, they are used for recognition with discrete Hidden Markov Model (HMM). Recognition stage is composed of two stages, namely training and classification. The Baum Welch algorithm is used for HMM training. In classification part the likelihood of each HMM is calculated and the HMM with the highest likelihood is chosen. In order to measure the success rate of the system, user-dependent and independent tests were conducted for 10 Turkish Sign Language gesture and the system is shown to be working with 85.8% accuracy in the user independent case and 100% in user dependent case.