Optimising the website accessibility conformance evaluation methodology

Hambley A., Yesilada Y., Vigo M., Harper S.

19th International Web for All Conference, W4A 2022, Virtual, Online, France, 25 - 26 April 2022 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1145/3493612.3520452
  • City: Virtual, Online
  • Country: France
  • Keywords: Web, Accessibility, Automated evaluation, Demographics, Sampling
  • Middle East Technical University Affiliated: Yes


© 2022 ACM.The Website Accessibility Conformance Evaluation Methodology (WCAG-EM) is a methodology that guides an auditor through evaluating a website's conformance to the Web Content Accessibility Guidelines (WCAG). In this paper, we raise some concerns present in the five stages of the WCAG-EM, including the evaluative scope, the non-probabilistic sampling approach, and the potential for bias within the selected sample. We propose a novel framework positioned parallel to the WCAG-EM, but with additional statistical-based metrics to address these concerns. These metrics consider the coverage, representativeness, complexity, popularity, freshness, and accessibility of a population of pages. We provide an overview of these metrics and discuss the possible use with three web page population-sourcing methods: server log files, breadth-first crawling, and depth-first crawling. Finally, we highlight the potential of utilising server log files as a population-sourcing method.