Want to fully comply with the EAA? Your accessibility scanner may not be as reliable as you think.
- Jul 7
- 3 min read
Updated: Jul 8
Automated accessibility scanners catch only a fraction of real issues. Here's what they miss, why it matters for EAA compliance, and a free toolkit to test properly.
Run an automated accessibility scanner over your website and you'll get a reassuring-looking report. A score. A list of issues, colour-coded and tidy. Fix them, and the score climbs. Job done. Except it isn't, and if you're relying on that report as evidence of European Accessibility Act compliance, it could give you a false sense of security.
The 30% problem
Automated accessibility scanners like axe DevTools, WAVE, Lighthouse, Siteimprove etc are genuinely useful. They're fast, consistent, and excellent at catching a certain class of problem: insufficient colour contrast, missing image descriptions, absent form labels, broken heading structures.
But study after study puts the same ceiling on them. Automated tools reliably detect only around 30–40% of real accessibility barriers. The other 60–70% require human judgement, because they're about meaning and experience, not just code.
A scanner can confirm a link has text. It can't tell you that "click here," repeated eleven times on the page, is useless to someone navigating by screen reader. It can confirm a form field has a label. It can't tell you that the error message which appears when you get something wrong is invisible to assistive technology. It can confirm a button exists. It can't tell you that you physically cannot reach that button using only a keyboard which, for the many people who navigate that way, means the task simply cannot be completed.
Those are the sort of barriers that generate complaints. Those are the sort of barriers that are showing up in the lawsuits now being filed across the EU. And they're precisely the ones a scan-only approach will miss.
What proper testing actually involves
Meaningful accessibility testing uses three techniques together, because each catches what the others miss:
Automated scanning: for broad, fast coverage of the mechanical checks. This is your starting point, not your finish line.
Manual expert review: a person checking each requirement against the real page. Asking questions like - does this actually make sense, work, and communicate clearly?
Assistive technology walkthroughs: testing genuine user journeys (checkout, sign-up, booking) with a screen reader, keyboard-only, voice control and magnification. This is best done by disabled users so they can tell you their actual experience of using your digital service.
The standard that underpins all of this for EAA purposes is EN 301 549, the harmonised European accessibility standard, which references WCAG 2.2 at Levels A and AA, covering 55 individual success criteria for web content, plus further requirements for apps, documents, and physical terminals.
That's a lot to work through methodically. Which is exactly why we built a tool to simplify and structure it for you.
The SIAS Digital & Technical Conformance Assessment Toolkit is the same workbook we use to audit client websites and apps — and we've made it free to download.
It's a single Excel workbook that gives you:
The full WCAG 2.2 checklist: all 55 Level A and AA success criteria, each explained in plain English rather than standards-body jargon, and mapped to the relevant part of EN 301 549.
Addendums for apps, documents, hardware and support: because accessibility doesn't stop at your website.
An automated-scan import tab: paste in your scanner's results and it automatically links each finding to the right checklist item, so you can see instantly what's already flagged and focus your manual effort where it counts.
A manual and assistive-technology test log: a structured place to record keyboard, screen reader, voice control and magnification testing.
An auto-calculating conformance dashboard: your overall score, a breakdown by accessibility principle, and your highest-severity issues highlighted automatically, so you know what to fix first.
An accessibility statement draft: a head start on the public statement the EAA requires, populated from your own results.
This toolkit is built for the people who actually have to do the work — in-house developers, QA teams, and agencies who need a repeatable process rather than a one-off scanner report.
It won't do the testing for you. But it will make sure that when you test, you test the whole picture and not just the 30% a machine can see.
If you'd rather not do the assessment audit yourself, SIAS can do it for you — automated scanning, manual review, assistive-technology testing and a prioritised remediation roadmap, delivered as one complete package. Get in touch to talk it through.
Comments