A comparative analysis of single-page and site-wide tools with the OPTIMAL-EM framework

Creative Commons License

Hambley A., YILMAZ Y., Vigo M., Harper S.

Universal Access in the Information Society, vol.25, no.3, 2026 (SCI-Expanded, SSCI, Scopus)

  • Publication Type: Article / Article
  • Volume: 25 Issue: 3
  • Publication Date: 2026
  • Doi Number: 10.1007/s10209-026-01356-1
  • Journal Name: Universal Access in the Information Society
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Scopus, IBZ Online, Aerospace Database, Compendex, INSPEC, Library, Information Science & Technology Abstracts (LISTA), Information Science & Technology Abstracts (LISTA), Technology Collection (ProQuest)
  • Keywords: Accessibility, Accessibility metrics, Automated evaluation, OPTIMAL-EM, Web accessibility evaluation
  • Open Archive Collection: AVESIS Open Access Collection
  • Middle East Technical University Affiliated: Yes

Abstract

Web accessibility auditing is critical for ensuring websites are accessible to those with disabilities, and whilst automated tools cannot evaluate every accessibility barrier, they can help evaluate specific barriers and illustrate the accessibility of a page. This paper proposes applying our OPTIMAL-EM framework for evaluating web accessibility evaluation tools, comprising the following six metrics to conduct a comparative analysis of single-page and site-wide accessibility evaluation tools: coverage, complexity, accessibility, representativeness, popularity and freshness. Our comparative analysis identified gaps in the current tooling regarding popularity and representativeness and recommends incorporating a method of prioritising web pages and support for representative sampling to make web accessibility evaluation tools more effective and efficient. Specifically, we identify the absence of representativeness measures in the current suite of accessibility tools, a lack of support for regional accessibility standards, and the potential use of popularity measures to optimise the web accessibility evaluation process. The main contribution of this paper is to demonstrate how evaluation tools can be systematically assessed with our OPTIMAL-EM framework.