A method for isolated sign recognition with Kinect

---

Tezin Türü: Yüksek Lisans

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü, Türkiye

Tezin Onay Tarihi: 2014

Öğrenci: EMRE IŞIKLIGİL

Danışman: SİNAN KALKAN

Özet:

Although there are various studies on sign language recognition (SLR), most of them use accessories like coloured gloves and accelerometers for data acquisition or require complex environmental setup to operate. In my thesis, I will use only Microsoft Kinect sensor for acquiring data for SLR. Kinect lets us obtain 3D positions of the body joints in real time without the help of any other device. After an isolated sign is captured, paths of the discriminative body joints are extracted. Then, a vector consisting of the extracted paths, called Sign Graph, is created to describe the isolated sign. To be able to compare two sign graphs, as the distance metric, I propose using the average warping distance of the joint paths that the sign graphs include. Dynamic Time Warping is used for effective calculation of the warping distance. Once a distance measure is defined between Sign Graphs, they are classified using k Nearest Neighbours algorithm. The proposed method performed better than the state of the art and achieved recognition rate of 59.3% in signer-independent experiments and 91.0% in signer-dependent experiments with a dataset consisting of 40 signs obtained from 13 different signers.