Parallel and Distributed Architecture for Multilingual Open Source Intelligence Systems

Karamanlioglu A., Yurtalan G., Karatas Y. B.

Proceedings of the 17th European Conference on Software Architecture, ECSA 2023, İstanbul, Türkiye, 18 - 22 Eylül 2023, cilt.14590 LNCS, ss.438-450 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 14590 LNCS
  • Doi Numarası: 10.1007/978-3-031-66326-0_27
  • Basıldığı Şehir: İstanbul
  • Basıldığı Ülke: Türkiye
  • Sayfa Sayıları: ss.438-450
  • Anahtar Kelimeler: data scraping, distributed systems, multilingual data processing, open source intelligence, OSINT architecture, parallel architecture
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

The proliferation of publicly available information across multiple languages presents both unique challenges and opportunities for Open Source Intelligence (OSINT) systems. This paper proposes a novel architecture for multilingual OSINT that is both parallel and distributed. The architecture integrates language identification and translation capabilities, enabling it to handle linguistically diverse data by transforming it into a unified format for efficient analysis. Designed specifically to address the challenges of parallel and distributed processing in OSINT systems, this architecture aims to offer scalability and performance benefits when dealing with massive data volumes. Our primary focus has been on devising strategies and tactics that address these concerns, providing a robust solution for the collection, processing and analysis of data in various languages. This work marks a significant step towards the development of more globally inclusive OSINT systems.