Multi-sensory perception in virtual world animation using McGurk effect

---

Tezin Türü: Yüksek Lisans

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Enformatik Enstitüsü, Modelleme ve Simülasyon Anabilim Dalı, Türkiye

Tezin Onay Tarihi: 2019

Tezin Dili: İngilizce

Öğrenci: BABAK ALLAHGHOLIPOUR

Danışman: Cengiz Acartürk

Özet:

Information about multisensory perception and brain cognition mechanism lead to a better virtual world creation. Brain store visual and auditory information to use in predicting future events. Image memory takes information from visual clues which during speech recognition happens by recording visual information and its auditory mappings. Based on this process, brain create prediction mechanism for future events. In speech recognition brain record sound and its visual presentation. When brain encounter difficulty in perception an auditory stimulus, it looks for visual mappings to guess the result. Considering the fact that some completely unrelated similarity may exist, some mapping may not lead to better understanding the world around. When previous information misleads the brain to understand the situation cognitive biases happen. McGurk Effect discusses how different lip movements affect auditory perception to get disparate information. Variety of factors influence amount of dependency on visual clues. Tracking speaking, especially in challenging environments, can be increased by visual clues such as face expressions, tongue and lips movements. Considering these elements in virtual world and game designing lead to improving believability. Perception processes, brain nature, designing factors, graphical elements, and visual structures are discussed to find out improved ways in designing realistic auditory and visual components.