YKSBench: Stress-Testing Multimodal Models with Exam-Style Questions
Conference on Neural Information Processing Systems (NeurIPS), California, Amerika Birleşik Devletleri, 2 - 07 Aralık 2025, ss.1-20, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Basıldığı Şehir: California
- Basıldığı Ülke: Amerika Birleşik Devletleri
- Sayfa Sayıları: ss.1-20
- Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
- Orta Doğu Teknik Üniversitesi Adresli: Evet
Özet
Large Multimodal Models (LMMs) have made remarkable progress in bridging language and vision, yet their performance on visually grounded scientific and exam-style reasoning tasks remains far below human-level ability. To systematically probe these limitations, we introduce YKSBench, a multimodal benchmark of 2,047 university entrance exam questions spanning mathematics, geometry, physics, chemistry, biology, and geography. Unlike prior benchmarks that focus narrowly on mathematics or synthetic tasks, YKSBench captures diverse question formats where visual information is indispensable. Despite the apparent simplicity of many problems to humans, state-of-the-art LMMs show striking deficiencies: the best-performing proprietary model, GPT-5, reaches only 39.34% accuracy, while the strongest open-source model, Gemma 3 27B, achieves 26.82% accuracy. We further provide qualitative analyses and novel scientific figures illustrating failure modes where models misinterpret diagrams, schematics, or spatial layouts. Positioned as a complementary resource to existing benchmarks such as MathVista, MathVision, and MMStar, YKSBench broadens the evaluation landscape and emphasizes the urgent need for improved visual reasoning in LMMs. Dataset is open-sourced at: https://huggingface.co/datasets/metu-yks/yksbench.