If you are setting up accessibility testing for the first time, the most expensive mistake is treating an automated scanner as the finish line. Automated tools are fast, cheap, and repeatable, but they can only evaluate rules a machine can measure, which leaves a large share of WCAG criteria untested. The rest live in places a machine cannot judge: whether your alt text is meaningful, whether a custom dropdown works with a keyboard, whether a screen reader announces the right thing.
The honest framing for automated vs manual accessibility testing is not which is better, but which job each does. Automated checks scale across thousands of pages and catch regressions on every commit. Manual and assistive-technology testing catch the issues that actually block real users. This guide explains where the line falls and how to combine both into one workflow that holds up against WCAG 2.2 and the regulators who now reference it.
What automated accessibility tools reliably catch
Automated engines parse the rendered DOM and computed styles, then apply deterministic rules. They are genuinely good at anything that can be expressed as a measurable condition, and you should lean on them hard for these:
- Color contrast that fails a numeric threshold: text below 4.5:1 (or below 3:1 for large text, meaning at least 18pt or 14pt bold) and UI components or graphics below 3:1, per WCAG 1.4.3 and 1.4.11.
- Missing programmatic attributes: an img with no alt attribute, a form field with no associated label, a button with no accessible name.
- Document structure problems: missing page language, empty headings, duplicate IDs, or skipped heading levels in the source order.
- Invalid ARIA: roles that do not exist, required ARIA attributes that are absent, or aria-* references pointing to IDs that are not on the page.
- Some keyboard signals: positive tabindex values, or elements with click handlers but no focusable role.
These are real defects, and finding them on every page automatically is exactly what scanners are for. A free checker like AccessScan will surface this class of issue in seconds, which is why automated testing belongs in your pipeline first. You can run a page through it at AccessScan.
What automated tools cannot catch, and why
The gap is not about tool quality. It is that most accessibility is about meaning and behavior, which a rule engine cannot evaluate. A scanner confirms an alt attribute exists; it cannot confirm the text describes the image. alt="image" and alt="" both pass the presence check, yet one is useless and the other correctly hides decoration. Judging that requires a human reading it in context.
The same blind spot runs across the four WCAG principles, Perceivable, Operable, Understandable, and Robust:
- Operable: whether a custom modal traps focus, whether you can complete a multi-step form with only the keyboard, and whether the visible focus indicator is actually visible. WCAG 2.2 added 2.4.11 Focus Not Obscured (Minimum, AA), 2.5.7 Dragging Movements (AA), and 2.5.8 Target Size Minimum (AA), all of which need real interaction to verify.
- Understandable: whether error messages explain how to fix the problem, whether help sits in a consistent place (3.2.6 Consistent Help, Level A), and whether 3.3.8 Accessible Authentication (AA) is met without forcing a cognitive memory test.
- Robust: whether a screen reader announces a dynamic update, a sortable table, or a live region in a way that makes sense. A scanner sees valid ARIA; only a screen reader reveals whether it communicates.
WCAG 2.2 also obsoleted 4.1.1 Parsing, so older tools that still flag duplicate-ID parsing errors as failures are now reporting noise. A tool that over-reports erodes trust as much as one that under-reports, so keep your rule sets current.
Why manual and assistive-tech testing is non-negotiable
Two methods cover almost everything automation misses. The first is a structured manual pass driven by a written list of criteria, so coverage stays consistent run to run; a working accessibility checklist keeps reviewers honest. The second is testing with the assistive technology your users actually rely on.
Keyboard testing is the highest-value manual step and takes minutes. Set the mouse aside and Tab through the page: every interactive element should be reachable, in a logical order, with a clearly visible focus ring, and nothing should trap you. Then test with at least one screen reader (NVDA on Windows and VoiceOver on macOS are both free) to hear what users hear. Custom widgets and dynamic content are where automated and manual results diverge the most, so prioritize those.
How to combine automated and manual testing
Sequence the two so each does what it is best at. A workflow that scales looks like this:
- Automate the wide pass. Run a scanner in CI on every pull request to catch contrast, missing names, and ARIA regressions before they ship. This is your fast, repeatable floor.
- Manually test the critical paths. For each key flow, such as sign-up, checkout, and search, do a keyboard pass and a screen-reader pass. You do not need to test every page, but you must cover every distinct pattern and template.
- Audit deeply before milestones. Before a release or a compliance deadline, run a full review against WCAG 2.2 Level A and AA, combining tool output, manual review, and assistive-tech findings into one report.
- Document the result. Record what you tested and what is still outstanding in an accessibility statement, which most accessibility laws expect you to publish.
This matters now because the European Accessibility Act (Directive (EU) 2019/882) applies from 28 June 2025, with a baseline of WCAG 2.2 Level A and AA referenced through EN 301 549. The earlier EU Web Accessibility Directive (2016/2102) already holds public-sector bodies to the same standard. Penalties for non-compliance vary by country, but enforcement is real, and an automated scan alone is not evidence of conformance. Regulators and the success criteria themselves assume human evaluation.