Deep Multi-Object Symbol Learning with Self-Attention Based Predictors


Ahmetoglu A., Oztop E., UĞUR E.

31st IEEE Conference on Signal Processing and Communications Applications (SIU), İstanbul, Türkiye, 5 - 08 Temmuz 2023, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/siu59756.2023.10223865
  • Basıldığı Şehir: İstanbul
  • Basıldığı Ülke: Türkiye
  • Orta Doğu Teknik Üniversitesi Adresli: Hayır

Özet

This paper proposes an architecture that can learn symbolic representations from the continuous sensorimotor experience of a robot interacting with a varying number of objects. Unlike previous works, this work aims to remove constraints on the learned symbols such as a fixed number of interacted objects or pre-defined symbolic structures. The proposed architecture can learn symbols for single objects and relations between them in a unified manner. The architecture is an encoder-decoder network with a binary activation layer followed by self-attention layers. Experiments are conducted in a robotic manipulation setup with a varying number of objects. Results showed that the robot successfully encodes the interaction dynamics between a varying number of objects using the discovered symbols. We also showed that the discovered symbols can be used for planning to reach symbolic goal states by training a higher-level neural network.