Compare
Personaudit vs pa11y-ci
pa11y-ci tests the pages you name against a limit. Personaudit remembers the defects you already have and fails on new ones.
Last reviewed 28 October 2026
As of 2026-10-28. Claims about pa11y-ci come from its README on that date. Check the README before you rely on a row, because both tools change.
The short version
pa11y-ci is a mature command line runner. You describe the pages in a config file or point it at a sitemap, set a threshold, and it fails when a page goes over. It is a good fit when you know exactly which URLs matter.
Personaudit finds pages by crawling and judges a run against a baseline. A page that already has ten known defects can keep them, and the build fails when an eleventh appears.
Side by side
| Topic | pa11y-ci | Personaudit |
|---|---|---|
| License | LGPL-3.0-only | MIT |
| Configuration | A config file, or one you point to with --config (JSON or JavaScript). | CLI flags and GitHub Action inputs. There is no config file for the URL list. |
| Which URLs get tested | The URLs you list, or every URL in a sitemap passed with --sitemap. --sitemap-exclude filters by regular expression. | A crawl from one start URL, capped by max-pages. Personaudit does not read a sitemap. |
| Runners | htmlcs (HTML_CodeSniffer, the default), axe, or both. | axe-core only. |
| What fails the build | A page fails when its issue count is above its threshold. Thresholds can be set in defaults or per URL. | Defects not in a committed baseline, at or above the severity you set with fail-on. |
| Telling the same defect apart across runs | The README describes issue counts per page, not defect identity. | A defect key built from the axe rule and a selector with framework-generated ids collapsed. |
| Accepting a known defect | Raise the page's threshold. | A suppressions file where each entry has a reason, an owner and an expiry date. An expired entry on a still-present defect fails the gate. |
| Machine-readable output | A JSON reporter, to the console or to a file you name. | scan.json next to a Markdown report. |
| Pull request feedback | Not described in the README. | One sticky pull request comment with new, fixed and still-open counts, when comment is true. |
Where pa11y-ci is better
- It reads a sitemap, so a large site is covered without listing URLs by hand. Personaudit does not ingest a sitemap.
- A config file keeps the URL list, defaults and per-URL options in version control. Personaudit has no equivalent file.
- Thresholds are set per URL, so a page you know is rough can have its own limit.
- It offers HTML_CodeSniffer as a runner, alone or alongside axe. Personaudit runs axe-core only.
Where Personaudit fits
- A count threshold lets one old defect be swapped for a new one without failing. A baseline keyed by defect catches the swap.
- You want accepted defects to expire. A suppression carries a reason, an owner and a date, and fails the gate once the date passes.
- You want the pull request to carry the result as one comment that updates in place.
- The pages you care about sit behind a login. Pa11y can script login steps with its actions option; with the Personaudit CLI you save a session once and the crawl reuses it.
Automated checks, from either tool, catch only part of what WCAG covers. A clean run does not mean the site works for everyone. See how Personaudit tests and the CI gate guide.
Sources
- pa11y/pa11y-ci README, read 2026-10-28.
- Personaudit facts come from this project's own source: the action inputs in
action.yml, the defect key insrc/agent/defect-key.ts, and the suppression rules insrc/crawler/suppressions.ts.