Low-Level Hardware Requirement Classification Using Large Language Models: Challenges, Insights, and Future Directions for Embedded Control Systems


Uyar E. B., Gürsoy A. E., Gökçe C., TAŞKAYA TEMİZEL T.

Joint of REFSQ 2025 Workshops, Doctoral Symposium, Posters and Tools Track, and Education and Training Track, REFSQ 2025, Barcelona, İspanya, 7 - 10 Nisan 2025, cilt.3959, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 3959
  • Basıldığı Şehir: Barcelona
  • Basıldığı Ülke: İspanya
  • Anahtar Kelimeler: Embedded Control Systems, Large Language Models, Requirements Classification
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

Automated Requirements Engineering (RE) activities can streamline development processes, reduce errors, and facilitate informed decision-making, particularly for low-level hardware requirements where modifications are costly. Classification is a widely studied automated RE activity for software requirements. Yet, its applicability remains underexplored due to the lack of structured datasets. This study adapts and evaluates software requirement classification techniques for hardware by extracting low-level requirements from open-source hardware design artifacts of Embedded Control Systems. We evaluate two classification methods: fine-tuning a BERT-based model and zero-shot prompting with a quantized LLM (Qwen2.5). While fine-tuning achieved high accuracy, zero-shot classification with specific prompts outperformed it in overall performance, achieving an average F1-score of up to 90% on the hold-out test set. Our findings suggest that automating downstream RE activities for low-level hardware requirements may not require large, task-specific datasets; however, classification performance can be further improved and can serve as an enabler for advanced tasks.