Multimodal communication in instructional settings: An investigation of the functional roles of gestures and arrows

Creative Commons License

Acarturk C., Coskun M., EMİL S.

REVISTA SIGNOS, cilt.54, sa.107, ss.867-892, 2021 (AHCI) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 54 Sayı: 107
  • Basım Tarihi: 2021
  • Doi Numarası: 10.4067/s0718-09342021000300867
  • Dergi Adı: REVISTA SIGNOS
  • Derginin Tarandığı İndeksler: Arts and Humanities Citation Index (AHCI), Social Sciences Citation Index (SSCI), Scopus, IBZ Online, Periodicals Index Online, MLA - Modern Language Association Database, DIALNET
  • Sayfa Sayıları: ss.867-892
  • Anahtar Kelimeler: Modality, deixis, arrows, gestures, multimodality, TEXT, REPRESENTATIONS, DISCOURSE, CHILDREN, SPEECH
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

In daily communication and instructional settings, arrows and spontaneous gestures usually convey similar spatial information to interlocutors. They can visualize spatial information in 2D or 3D space based on shared conceptual and spatial representations. The present study investigates the relationship between spontaneous gestures and arrows from the perspective of generation and comprehension in instructional settings. We introduce a corpus of arrows in written multimodal text and a corpus of gestures generated during communication through multimodal instructional material. We also report an experimental investigation that used corpus material to study the impact of gestures on comprehension learning. A comparison of the corpora reveals functional similarities and differences between arrows and gestures and their complementary role in multimodal communication. The findings suggest that arrows and gestures may share a common conceptual space during communication, having a significant impact on comprehension and learning.