Automated Accessibility Testing Beyond Axe

What Axe-Core Does Well

Axe-core (and tools built on it, like Lighthouse, pa11y, and jest-axe) is the industry standard for automated WCAG testing. It excels at:

• Rule-based validation: Checks approximately 80+ WCAG rules covering images, forms, color contrast, document structure, and ARIA usage.
• Fast execution: Runs in milliseconds against a DOM, making it suitable for unit tests and CI.
• Browser integration: Works in Puppeteer, Playwright, Cypress, and browser DevTools (the Accessibility tab).
• Low false-positive rate: Rules are carefully tuned. When axe reports a violation, it is almost always a real issue.
• Industry adoption: Used by millions of developers, well-documented, and actively maintained.

If your project does not yet run axe-core in CI, start there. It catches the most common accessibility defects with minimal setup. The question this page addresses is: what happens after you pass axe?

What Rule-Based Tools Cannot Catch

Rule-based tools operate on a simple model: "Does this element violate a known WCAG success criterion?" This model has blind spots that affect real screen reader users.

Announcement Quality

A button can have an accessible name (passing all axe rules) but that name can be confusing, redundant, or overly verbose. Consider:

<!-- Passes axe: button has an accessible name -->
<button aria-label="Click here to submit the contact form to send your message">
  Submit
</button>

<!-- What NVDA announces: -->
<!-- "Click here to submit the contact form to send your message, button" -->
<!-- The visible text "Submit" is overridden by the verbose aria-label -->

Axe will pass this button because it has a name. But the name is redundant (it restates what is already visually obvious) and verbose (30 words where 1 would suffice). Speakable shows you the full announcement so you can judge quality, not just compliance.

Cross-Reader Differences

Axe checks against a single accessibility API model. It cannot tell you that VoiceOver says "dimmed" while NVDA says "unavailable" for the same disabled button, or that Narrator adds "to activate, press Enter" while others do not. These differences affect how users understand your interface:

npx @reticular/speakable disabled-button.html -f text -s all

# === NVDA ===
# Save changes, button, unavailable
#
# === VoiceOver ===
# Save changes, dimmed, button
#
# === Narrator ===
# Save changes, button, disabled

All three pass axe. But a developer writing documentation or training materials needs to know that users will hear different words depending on their reader. Speakable reveals these differences.

Verbosity and Redundancy

Some ARIA patterns cause screen readers to repeat information. A link inside a navigation landmark might announce as "navigation, list, list item, link, Home" when the user just wants to hear "Home, link". Axe does not flag this because no rule is violated. The semantics are technically correct but the experience is noisy.

Speakable shows you exactly how verbose the output is for each element, helping you identify where to simplify structure or remove redundant ARIA annotations.

Where Speakable Fills the Gap

Speakable and axe address different questions. Here is how they compare:

The tools are complementary. Axe catches the "is this valid?" question. Speakable catches the "what will users hear?" question. Running both gives you confidence in both compliance and user experience.

Using Both Together

The most robust CI pipeline runs both axe and Speakable on every pull request. Axe catches rule violations, Speakable catches speech output regressions. Here is a combined GitHub Actions workflow:

Combined GitHub Actions Workflow

name: Accessibility CI
on: [pull_request]

jobs:
  accessibility:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci

      # Step 1: Rule-based checks with axe
      - name: Run axe-core checks
        run: |
          npx playwright install --with-deps chromium
          npx playwright test --project=accessibility
        # Assumes you have Playwright tests that run axe on rendered pages

      # Step 2: Predictive screen reader checks with Speakable
      - name: Run Speakable audit
        run: |
          npx @reticular/speakable src/components/*.html -f audit
          # Exit code 1 if issues found (missing names, broken hierarchy)

      # Step 3: Regression check against baselines
      - name: Check for screen reader regressions
        run: |
          npx @reticular/speakable src/components/Button.html \
            --diff baselines/Button.baseline.txt -f text -s all
          # Exit code 2 if output differs from baseline

This workflow catches three categories of problems: WCAG rule violations (step 1), new accessibility defects in HTML (step 2), and regressions in existing screen reader output (step 3). A pull request must pass all three to merge.

Limitations of Both Approaches

Neither axe nor Speakable replaces manual testing with real assistive technology. Both operate on static representations (DOM for axe, HTML for Speakable) and cannot test:

• Keyboard interaction patterns: Focus traps, roving tabindex, escape key handling.
• Live region timing: Whether aria-live announcements interrupt appropriately or arrive too late.
• Mode switching: How NVDA/JAWS transition between browse mode and focus mode.
• Cognitive flow: Whether the overall page structure makes sense to navigate without vision.
• Speech synthesis behavior: Pronunciation of unusual words, handling of abbreviations, speech pauses.

Additionally, axe's scope is limited to approximately 57% of WCAG success criteria (per Deque's own documentation). Many criteria require human judgment: "Is this text meaningful?", "Is this instruction clear?", "Is this error message helpful?"

Speakable's scope is limited to static HTML analysis (unless you use the runtime engine for dynamic content). It produces heuristic predictions, not exact transcripts of any specific screen reader version. The predictions are strong for common patterns and may diverge for edge cases or very new screen reader features.

The recommended approach: run both tools in CI to catch what automation can catch, then schedule manual screen reader testing before each major release. This gives you fast feedback on every commit plus deep verification at key milestones.