Online data analysis and reduction: An important Co-design motif for extreme-scale computers

Foster I., Ainsworth M., Bessac J., Cappello F., Choi J., Di S., ...Daha Fazla

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, cilt.35, sa.6, ss.617-635, 2021 (SCI-Expanded, Scopus) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 35 Sayı: 6
  • Basım Tarihi: 2021
  • Doi Numarası: 10.1177/10943420211023549
  • Dergi Adı: INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, ABI/INFORM, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, Metadex, Civil Engineering Abstracts
  • Sayfa Sayıları: ss.617-635
  • Orta Doğu Teknik Üniversitesi Adresli: Hayır

Özet

A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.