Testing Ecosystem

How Speakable fits alongside other accessibility testing tools — and when to use each approach.

Static vs Runtime Testing

Accessibility testing tools fall into two categories. Understanding the difference helps you build a testing strategy that catches issues early without sacrificing confidence in the final product.

Static Analysis

Analyzes HTML structure without running a real screen reader. Fast, deterministic, runs anywhere. Catches structural issues like missing names, broken heading hierarchy, and ARIA problems.

Tools: Speakable, axe-core, HTML_CodeSniffer

Runtime Testing

Drives an actual screen reader against a live page. Catches dynamic interaction issues, focus management bugs, and live region behavior that static analysis can't see.

Tools: Guidepup, manual VoiceOver/NVDA testing

	Static (Speakable)	Runtime (Guidepup)
Speed	Milliseconds	Seconds to minutes
Setup	`npm install`, runs anywhere	OS-specific screen reader required
CI/CD	Any pipeline (Linux, macOS, Windows)	Requires macOS or Windows runners
Coverage	HTML structure, ARIA, names, roles, states	Dynamic behavior, focus, live regions
Accuracy	Heuristic (approximation)	Ground truth
Best for	Early detection, regression prevention	Final validation, complex interactions

Works Alongside Guidepup

Guidepup is a runtime screen reader automation library that drives real VoiceOver and NVDA instances programmatically. It's the closest you can get to automated real-user screen reader testing.

Speakable and Guidepup solve different problems at different stages:

Speakable during development

Run speakable page.html -f audit locally to catch structural issues before committing. No screen reader setup needed. Runs in milliseconds.

Speakable in CI

Automate regression detection with --diff on every PR. Fail builds when screen reader output changes unexpectedly. Works on any CI runner.

Guidepup for validation

Use Guidepup to validate critical user flows where dynamic behavior matters — focus management, live regions, keyboard navigation sequences. Requires macOS (VoiceOver) or Windows (NVDA) runners.

This layered approach catches the majority of screen reader issues through fast static analysis, then validates the critical paths with runtime tools where it counts.

Combined Workflow Example

Here's how a team might use both tools in a GitHub Actions pipeline:

.github/workflows/a11y.yml

name: Accessibility

on: [push, pull_request]

jobs:
  # Fast static checks — runs on any runner
  static-analysis:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - name: Speakable audit
        run: npx @reticular/speakable ./dist/index.html -f audit
      - name: Speakable regression check
        run: npx @reticular/speakable ./dist/index.html --diff ./baseline.html

  # Runtime validation — runs on macOS for VoiceOver
  runtime-validation:
    runs-on: macos-latest
    needs: static-analysis  # Only run if static checks pass
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - name: Guidepup VoiceOver tests
        run: npx guidepup test

Static analysis runs first on a cheap Linux runner. Runtime validation only runs if static checks pass, saving expensive macOS runner minutes for when they're actually needed.

When to Use What

Use Speakable when…

You're writing components, reviewing PRs, running CI checks, or need fast feedback on how screen readers will interpret your HTML structure.

Use Guidepup / real screen readers when…

You need to validate focus management, live region announcements, keyboard navigation flows, or any behavior that depends on JavaScript execution and real-time DOM updates.

Use axe-core when…

You need rule-based WCAG violation detection (color contrast, missing alt text, form label association). Axe checks rules; Speakable predicts what screen readers will actually say.

Speakable + Guidepup: A Practical Testing Pattern

The most effective accessibility testing strategy uses both tools on the same component. Speakable validates structure instantly (names, roles, hierarchy), then Guidepup validates the interaction experience with a real screen reader.

Example: Testing a modal dialog

A modal needs both correct structure (dialog role, label, heading) and correct behavior (focus trap, escape to close, focus return). Speakable catches the first category; Guidepup catches the second.

Step 1 — Speakable: structural validation (runs anywhere, milliseconds)

// tests/a11y/modal-structure.test.ts
import { execSync } from 'child_process';

describe('ConfirmDialog structure', () => {
  const output = execSync(
    'npx @reticular/speakable src/components/ConfirmDialog.html -s all -f text'
  ).toString();

  it('announces as a dialog with a label', () => {
    expect(output).toContain('Confirm deletion, dialog');
  });

  it('has a heading inside the dialog', () => {
    expect(output).toMatch(/heading level [12], Confirm deletion/);
  });

  it('has labeled action buttons', () => {
    expect(output).toContain('Cancel, button');
    expect(output).toContain('Delete, button');
  });

  it('does not announce the backdrop', () => {
    // Backdrop should be aria-hidden
    expect(output).not.toContain('overlay');
  });
});

Step 2 — Guidepup: interaction validation (requires macOS runner)

// tests/a11y/modal-interaction.test.ts
import { voiceOver } from '@guidepup/guidepup';
import { webkit } from '@playwright/test';

describe('ConfirmDialog interaction', () => {
  it('traps focus inside the dialog', async () => {
    const browser = await webkit.launch();
    const page = await browser.newPage();
    await page.goto('http://localhost:3000/checkout');

    // Open the modal
    await page.click('[data-testid="delete-btn"]');

    // Start VoiceOver
    await voiceOver.start();

    // Verify focus moved into the dialog
    const firstAnnouncement = await voiceOver.lastSpokenPhrase();
    expect(firstAnnouncement).toContain('Confirm deletion');

    // Tab through — should stay inside dialog
    await voiceOver.press('Tab');
    const cancelBtn = await voiceOver.lastSpokenPhrase();
    expect(cancelBtn).toContain('Cancel');

    await voiceOver.press('Tab');
    const deleteBtn = await voiceOver.lastSpokenPhrase();
    expect(deleteBtn).toContain('Delete');

    // Tab again — should wrap back (focus trap)
    await voiceOver.press('Tab');
    const wrapped = await voiceOver.lastSpokenPhrase();
    expect(wrapped).toContain('Cancel'); // wrapped back

    // Escape closes and returns focus
    await voiceOver.press('Escape');
    const returned = await voiceOver.lastSpokenPhrase();
    expect(returned).toContain('Delete item'); // original trigger

    await voiceOver.stop();
    await browser.close();
  });
});

Why both matter

Speakable catches:

Missing dialog role
Missing aria-labelledby
Unlabeled buttons (icon-only)
Backdrop not hidden from AT
Heading hierarchy inside modal

Guidepup catches:

Focus not moving to dialog on open
Focus trap not working (Tab escapes)
Escape key not closing the dialog
Focus not returning to trigger on close
Live region not announcing state changes

CI pipeline: layered execution

Run Speakable on every push (cheap, fast, any runner). Run Guidepup only on PRs targeting main (expensive, requires macOS). This gives you fast feedback on structure and thorough validation before merge.

.github/workflows/a11y-layered.yml

name: Accessibility (Layered)

on:
  push:
    branches: ['**']
  pull_request:
    branches: [main]

jobs:
  # Runs on every push — fast, any runner
  structure:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - name: Speakable structural checks
        run: |
          npx @reticular/speakable dist/**/*.html -f audit
          npx @reticular/speakable dist/index.html --diff baseline.html

  # Runs only on PRs to main — thorough, macOS runner
  interaction:
    if: github.event_name == 'pull_request'
    needs: structure
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run build && npm run start &
      - name: Guidepup VoiceOver tests
        run: npx playwright test tests/a11y/*-interaction*

When Speakable alone is enough

Not every component needs Guidepup. For static content, forms, and simple interactive elements, Speakable's structural validation covers the important cases:

Speakable only

• Static pages and content
• Navigation landmarks
• Form labels and error messages
• Heading hierarchy
• Image alt text
• Button and link names
• Table structure

Add Guidepup

• Modal focus trapping
• Combobox / autocomplete
• Drag and drop
• Single-page app routing
• Toast / notification live regions
• Tab panel keyboard switching
• Carousel navigation

Ready to set up your pipeline?

See how to integrate Speakable into GitHub Actions and GitLab CI.

CI/CD Setup Guide