Sketch Recognition for Interactive Game Experiences Using Neural Networks


Korkut E. H. , Sürer E.

Lecture Notes in Computer Science, Entertainment Computing – ICEC 2021, 20th IFIP TC 14 International Conference, ICEC 2021, Coimbra, Portugal, 2 - 05 November 2021, pp.393-491

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1007/978-3-030-89394-1_31
  • City: Coimbra
  • Country: Portugal
  • Page Numbers: pp.393-491

Abstract

Human freehand sketches can provide various scenarios to the interfaces with their intuitive, illustrative, and abstract nature. Although freehand sketches have been powerful tools for communication and have been studied in different contexts, their capacity to create compelling interactions in games is still under-explored. In this study, we present a new game based on sketch recognition. Specifically, we train various neural networks (Recurrent Neural Networks and Convolutional Neural Networks) and use different classification algorithms (Support Vector Machines and k-Nearest Neighbors) on sketches to create an interactive game interface where the player can contribute to the game by drawing. To measure usability, technology acceptance, immersion, and playfulness aspects, 18 participants played the game and answered the questionnaires composed of four different scales. Technical results and user tests demonstrate the capability and potential of sketch integration as a communication tool to construct an effective and responsive visual medium for novel interactive game experiences.

Human freehand sketches can provide various scenarios to the interfaces with their intuitive, illustrative, and abstract nature. Although freehand sketches have been powerful tools for communication and have been studied in different contexts, their capacity to create compelling interactions in games is still under-explored. In this study, we present a new game based on sketch recognition. Specifically, we train various neural networks (Recurrent Neural Networks and Convolutional Neural Networks) and use different classification algorithms (Support Vector Machines and k-Nearest Neighbors) on sketches to create an interactive game interface where the player can contribute to the game by drawing. To measure usability, technology acceptance, immersion, and playfulness aspects, 18 participants played the game and answered the questionnaires composed of four different scales. Technical results and user tests demonstrate the capability and potential of sketch integration as a communication tool to construct an effective and responsive visual medium for novel interactive game experiences.