Accessibility testing with Chromatic
Accessibility testing is currently in private beta. Request early access »
Chromatic builds on Storybook and axe to seamlessly integrate accessibility testing into your development workflow. You write stories to create test cases for your components and run accessibility checks locally within Storybook.
Then when you’re ready to merge, Chromatic automatically runs accessibility regression tests on all your stories through CI. It also provides a dashboard that offers a bird’s-eye view of WCAG issues across all your components.
Why component-level accessibility tests?
Traditional page-level tests fail to address accessibility at its source. For decades, the best practice was to navigate page by page and run accessibility scans along the way to compile an audit report. This method is too noisy. When issues are identified at the page level, the root cause often lies deep within a component. You have to sift through thousands of violations to determine which component caused the problem.
What’s more, page-level tests occur too late in the development cycle. Developers first build components and then assemble them into pages. One flawed component can spread issues throughout your app, making problems harder to identify and more time-consuming to fix.
Chromatic runs accessibility tests at the component-level, where issues are easier to identify and fix. By integrating accessibility testing into your development workflow, Chromatic helps you catch violations early, before they multiply across pages.
Why use regression-based accessibility testing?
Axe has long been the industry standard for running automated accessibility tests on HTML-based interfaces. These tests audit the rendered DOM against a set of heuristics based on WCAG rules and widely accepted best practices. They act as the first line of QA to catch blatant accessibility violations.
When you run axe, it gives you a list of all violations detected across the UI being tested. While this is incredibly helpful, it’s unlikely you’ll address all issues immediately. Teams often inherit accessibility debt that can’t be resolved overnight. They want to address these issues incrementally while releasing new features without adding more debt.
By tracking violations against a story’s baseline, Chromatic helps prevent regressions in pull requests while tracking debt so you can fix it separately at your own pace.
Accessibility testing workflow
By combining Chromatic and Storybook, you get instant accessibility feedback at every phase, from development to running checks on CI. Here’s how it works:
During development: Fast feedback in Storybook
Storybook’s Accessibility addon simplifies running axe on individual components. You can run accessibility tests for all your stories in the background. If there are any violations, the test will fail, and you will see a summary in the sidebar. Violating elements are highlighted in Storybook’s canvas, allowing you to pinpoint the exact problem areas. You also get detailed error descriptions and guidance to help resolve issues quickly.
In CI: Regression-checks with Chromatic
When you run Chromatic, it creates a static build of your Storybook and uploads it to our Capture Cloud. From there, Chromatic spins up a fleet of standardized browsers to load all your stories in parallel and run accessibility tests.
Chromatic tracks accessibility violations over time, establishing a baseline for each story. This helps separate pre-existing issues from new ones.
What is an accessibility baseline?
In Visual testing, a baseline is the last known “good” state of the story. Subsequent snapshots are compared to this baseline to identify visual changes.
In Accessibility testing, a baseline is a set of accessibility violations detected for a story. Subsequent snapshots are compared to this baseline to identify new violations or to confirm resolved issues. Here’s an example:
Build #1 identifies several violations for this story. By accepting the baseline, 17 violations (across 3 rules) are added to the story’s baseline.
After making code changes, you run Build #2. Chromatic finds 10 additional violations across 2 rules. This difference is shown on the test page. You can also view violations from previous builds by expanding the table.
If you accept this snapshot, these 10 new violations will be added to the baseline.
In subsequent builds, Chromatic takes a new snapshot of accessibility violations for each story and compares them against the baselines. If there are new or altered violations, Chromatic flags them for your review.
Why run accessibility tests in both Storybook and Chromatic?
Complementary testing approaches
Storybook provides developers with a fast feedback loop. You can test components—that you’re working on—during development and receive instant feedback on accessibility issues. This enables you to fix problems locally before committing code and to identify and address violations in real time as you work.
Chromatic provides ultimate coverage for teams by testing across all viewports, modes, and user preferences. It runs tests consistently in CI environments, tracks violations over time, and ensures no regressions slip through the cracks. This comprehensive approach safeguards your entire project’s accessibility standards.
Single run vs Regression tracking
Single run (Storybook) shows all violations at a specific point in time, offering a comprehensive view of your component’s current state. However, reviewing these results can be overwhelming, especially in large codebases with accessibility debt.
Regression tracking (Chromatic) tracks violations from commit to commit and only flags new or changed violations. This makes it easier to fix issues incrementally and visualizes progress over time.
For a complete example of regression tracking, see the What is an accessibility baseline section above.
Accountability and team workflow
The regression approach allows teams to distinguish between legacy accessibility debt and new issues, making the work more manageable. Progress becomes visible and measurable, which helps maintain momentum in your accessibility improvement efforts.
With regression tracking, you can easily identify which violations were introduced in a specific pull request. Much like adding tests to meet coverage thresholds, this ensures developers address accessibility issues in their pull requests to avoid adding to the accessibility debt.
Frequently asked questions
What are Web Content Accessibility Guidelines (WCAG)?
The Web Content Accessibility Guidelines are a set of internationally recognized standards developed by the World Wide Web Consortium (W3C) through their Web Accessibility Initiative (WAI). These guidelines provide a comprehensive framework for making web content more accessible to people with disabilities.
What is an accessibility violation?
Accessibility violation refers to an issue detected within a story where an element fails to meet accessibility standards or guidelines (such as WCAG). These violations indicate barriers that might prevent people with disabilities from using the website or application effectively.
Why might you want to accept accessibility violations?
TL;DR: You’ll want to accept violations if they’re part of necessary structural changes leading to an eventual fix.
When you run axe, it lists all accessibility violations found in the story. However in some cases, addressing all issues immediately may not be feasible. In that case, you’d accept the violations and fix them incrementally.
Baselines help track accessibility issues from commit to commit as you make code changes, showing what was fixed and identifying any new issues introduced.
How does Chromatic track each unique violation?
Chromatic identifies unique violations using three characteristics:
- Type of violation (see full classification)
- Specific DOM node (e.g.,
<a>,<button>,<div>) - Bounding box of this DOM node (e.g., x/y coordinates plus width and height)
This approach prevents false positives caused by inconsequential DOM structure changes.
What happens when I fix an accessibility issue?
Fixes are auto-accepted to the baseline, learn more »
Can I run Accessibility Tests with Playwright or Cypress?
No, Chromatic only offers Accessibility Testing for Storybook.
Can I choose to only run accessibility tests?
Not yet. parameters.chromatic.disableSnapshot will disable both visual and accessibility tests. We’re working on allowing you to disable them accessibility and visual tests individually.
How do I configure Chromatic in CI for Accessibility Tests?
If you have already integrated Chromatic in your CI setup, no changes are needed for accessibility tests to work.
If you haven’t integrated Chromatic in your CI setup, please follow this guide.
Does Chromatic report “Needs Review” or “Incomplete” checks?
No, Chromatic does not report these checks. There are instances where axe-core is unable to accurately determine a violation. For example, it can’t accurately assess color contrast when text is rendered over a gradient or background image. You will see these issues reported in Storybook’s Accessibility panel under the “Incomplete” tab. However, Chromatic only reports on violations it can be certain about.
Does TurboSnap support accessibility tests?
Yes. TurboSnap analyzes your project’s Git history and dependency graph to identify which components and their dependencies have changed. It then only snapshots and runs visual and accessibility tests for stories associated with those changes. For the rest, it copies over the snapshots from baselines that didn’t change.
Do accessibility tests support modes?
Yes, if you’ve configured modes for a story, Chromatic will run accessibility tests on each mode too.”
Do accessibility tests support different browsers?
No, accessibility tests are only executed in Chrome.
Do accessibility tests support viewports?
Yes. Similar to how it works for visual tests, if you’ve set up viewports for a story, accessibility tests will run for each viewport.
Does Chromatic check for prefers-reduced-motion violations?
Not yet
Troubleshooting
Why are ignored regions not working with Accessibility Tests?
Ignored regions are a feature specific to visual tests and aren’t supported for accessibility and interaction tests.
I expected certain accessibility violation to be flagged—such as missing form input labels, tables without proper headers—but I’m not seeing them?
Ensure that the rule is enabled in your Storybook A11y addon configuration. If it is enabled, verify whether the check is running in Storybook.
While you might expect a violation, the check could actually be passing. For example, consider:
<input type="text" placeholder="something" />
You might expect a “missing form input labels” violation, but the placeholder attribute can provide text inputs with an accessible name. Similarly, table header rules only apply if the table has more than three rows or columns.
Why am I seeing different accessibility test results in Storybook vs Chromatic?
Storybook shows all violations at a specific point in time. Where are as, Chromatic tracks violations from commit to commit and only flags new or changed violations.
For more on how to combine the two into an effective workflow, refer to this section.
Why is Chromatic not reporting accessibility in the iFrame nested within my story?
Currently Chromatic does not support checking accessibility violations within iframes (nested within a story). That’s because axe is only loaded at the story level and not within the nested iFrames.