Web accessibility metrics are an invaluable tool for researchers, developers, governmental agencies and end users. Accessibility metrics help indicate the accessibility level of websites, including the accessibility level of individual websites, or even large-scale surveys of the accessibility of many websites. Recently, a plethora of metrics has been released to complement the A, AA, and AAA Levels measurement used by the WAI guidelines. However, the validity and reliability of most of these metrics are unknown and those making use of them are taking the risk of using inappropriate metrics. In order to address these concerns, this note provides a framework that considers validity, reliability, sensitivity, adequacy and complexity as the main qualities that a metric should have.
Well we undertook a study of validity and reliability of WCAG 2.0 and found that an 80% target for agreement is not attainable, when audits are conducted without communication between evaluators. Even with experienced evaluators the error rate is relatively high; and further, untrained accessibility auditors -be they developers or quality testers from other domains- do much worse than this. Read the full published text via ACM Author-izer Open Access on the publications page.
The last year has been an exciting one for SIGWEB. From the sponsorship of the large Web Science Conference, to that of the small Social Network Working Group; from the increase in student travel sponsorship (to $25,000pa), to our increasing volunteer effort; the SIG has been at work at all levels of our domain.
‘Caring for a person with Alzheimer’s can feel like a prison term spent in solitary confinement.’