OCR-mediated modality dominance in vision–language models: implications for radiology AI trustworthiness
Journal of Medical Imaging, cilt.13, sa.4, ss.45501, 2026 (ESCI, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 13 Sayı: 4
- Basım Tarihi: 2026
- Doi Numarası: 10.1117/1.jmi.13.4.045501
- Dergi Adı: Journal of Medical Imaging
- Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), Scopus, Compendex, EMBASE, INSPEC
- Sayfa Sayıları: ss.45501
- Anahtar Kelimeler: adversarial robustness, artificial intelligence trustworthiness, automation bias, medical imaging, multimodal aI safety, prompt injection, radiology informatics, vision–language models
- Orta Doğu Teknik Üniversitesi Adresli: Hayır
Özet
– Purpose – Vision–language models (VLMs) are increasingly proposed for radiologic decision support, yet the security implications of deploying optical character recognition (OCR)-capable models in diagnostic workflows remain poorly characterized. When image-embedded text is not treated as untrusted input, the visual channel becomes vulnerable to adversarial manipulation. Approach – Ten VLMs, none validated for clinical diagnosis, were evaluated on 600 brain magnetic resonance imaging studies for binary tumor detection under 5 conditions: clean input, visible radiology report injection, human-imperceptible stealth OCR injection, and multi-stage immune-prompt defense. A total of 27, 000 inference calls were analyzed. Results – At baseline, performance was heterogeneous, with a median accuracy of 0.69, a sensitivity of 0.79, and a specificity of 0.59. Visible injection caused universal specificity collapse to 0.00 across all models with a false-positive rate (FPR) of 1.00 and a median attack success rate (ASR) of 0.97. Stealth injection, despite being imperceptible to human reviewers, drove substantial degradation with a median accuracy of 0.43, an ASR of 0.57, and an FPR of 0.84. Immune prompting achieved only partial mitigation: under stealth injection, median ASR decreased to 0.44, and accuracy improved to 0.56, yet residual overcalling persisted with a median FPR of 0.67, and three models maintained an FPR of 1.00. Conclusions – Commercial VLMs exhibit a deployment-critical failure mode in radiology-like scenarios: OCR-readable text embedded in images can override pixel-level evidence, even under stealth conditions that evade human inspection. Prompt-level defenses provide insufficient protection. Any clinical integration of VLMs must be governed by system-level safeguards, including OCR-aware input handling, provenance controls, and enforced human verification, before deployment in safety-sensitive environments.