Accessibility Testing
Scope
This page serves as a wiki for the accessibility testing tools we have used, primarily during our work on the OpenIMIS and OpenLMIS projects. Here, you'll find an overview of various tools, their strengths, how they can help improve web accessibility, and our recommendations based on that experience.
Overview
ADOPT
Proven technologies we have high confidence in; our default choices.
ASSESS
Interesting technologies requiring investigation.
Technologies Breakdown
WAVE ADOPT
Recommended tool for automated accessibility testing. Use its browser extension for visual feedback directly on the page. Focus on reviewing the identified issues and utilizing the linked documentation.
WAVE provides valuable visual feedback highlighting elements with identified issues directly on the page. While the interface can seem busy initially, its key strength is its excellent documentation, which clearly explains detected problems, specifies the broken WCAG guidelines, and offers actionable insights. Based on our project experience, WAVE is a solid choice for automated accessibility testing and should be part of the standard toolkit.
Accessibility Insights for Web ADOPT
Primarily recommended for its structured manual testing guidance via the "Assessment" mode. Also provides automated checks and assisted checks (e.g., tab stops). Use Assessment mode for thorough manual evaluations.
This tool excels at guiding manual accessibility testing. Its Assessment mode provides clear step-by-step instructions on how to test specific criteria, what to look for, and expected outcomes. It also helps simulate different conditions. While it offers automated checks, WAVE generally provides more detailed insights in that specific area. Its strength lies in facilitating comprehensive manual testing, making it a great addition to the workflow.
BrowserStack TRIAL
Use opportunistically for automated accessibility testing, primarily leveraging the limited free tier scans. Do not rely on it as the sole automated checker due to free tier limitations and the high cost of the premium tier.
BrowserStack provides good automated accessibility testing capabilities. The premium tier offers detailed insights but comes at a high cost we don't believe is justified by the added value compared to alternatives. The free tier is useful but limited. Therefore, we recommend using BrowserStack's free scans when available but using WAVE (which provides similar issue detection) as the primary, reliable alternative for automated checks.
Lighthouse HOLD
Avoid using the Lighthouse accessibility audit as a primary tool for accessibility testing. Use dedicated tools like WAVE or Accessibility Insights instead.
Similar to Axe, Lighthouse didn't detect many accessibility issues in our testing, and the insights were average. While its integration into Chrome DevTools is convenient, this doesn't compensate for the lack of depth. For thorough accessibility testing, WAVE (automated) and Accessibility Insights (manual) offer far better value and should be prioritized.
Axe Accessibility HOLD
Avoid using Axe for automated accessibility testing in our projects. Prefer WAVE or the free tier of BrowserStack.
In our experience on OpenIMIS/OpenLMIS, Axe Accessibility (including the Pro version) offered less comprehensive insights and a less satisfactory user experience compared to alternatives like WAVE. Given that other tools provide more detailed reports and better testing capabilities, we do not recommend using Axe.