Creating a Generic Hand and Finger Gesture Recognizer by Using Forearm Muscle Activity Signals

---

Tezin Türü: Yüksek Lisans

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Enformatik Enstitüsü, Modelleme ve Simülasyon Anabilim Dalı, Türkiye

Tezin Onay Tarihi: 2017

Tezin Dili: İngilizce

Öğrenci: Umut Demirel

Asıl Danışman (Eş Danışmanlı Tezler İçin): Hüseyin Hacıhabiboğlu

Eş Danışman: Elif Sürer

Özet:

Hand and finger gestures are one of the most natural ways of non-verbal communication. Apart from their daily use in different cultures, they are also widely used in human-computer interaction. There are a variety of applications using gestures as inputs such as sign language recognition, robot control, mobile phone control, medical device control and video game control. Advances in near-field wireless communications made it possible to design and deploy low-cost, inconspicuous control devices which can be used to detect certain predefined hand gestures for use in interaction. This thesis aims to investigate and develop a generic hand and finger gesture recognizer by processing forearm muscle activity signals from such a device which consists of eight electromyography (EMG) and Inertial Measurement Unit (IMU) sensors. Two main presuppositions of this thesis is that i) the muscle activity on the forearm is a spatiotemporally bandlimited circular signal and that can be sampled using a finite number of sensors, and ii) different gestures result in different but consistent patterns which are separable. The approach used in this thesis is based on the extraction of features by joint processing of signals obtained from a commercially available, low-cost EMG armband and classification of the gestures by simple and low-complexity artificial neural networks (ANNs). The dictionary of gestures were chosen from a canonical catalog of expressive gestures of classical orchestra conductors. Two experiments were carried out to assess the system performance: i) thirteen different hand and finger gestures and one rest gesture are performed 5 times in the same session by 10 different subjects, and ii) data was collected in an ecological study from a conductor during a practice session of a symphony orchestra. It was found that the proposed method achieved an average classification ac- curacy of 63.14% (maximum of 79.87%) when the data distribution for train, test and validate parts of ANN used in classification process is separated by sessions. An average classification accuracy of 96.09% (maximum of 98.8%) was achieved when data distribution is random. Lastly, random data distribution of the five different gestures and one rest gesture data collected in an ecological study from a conductor resulted in 96.9% accuracy. All results were obtained with session and subject dependent experiments.