Testing¶

Run tests and validate changes before submitting pull requests.

Preflight Validation¶

Use the canonical validator before larger refactors and before preparing a PR. It runs fast repository checks, version consistency, YAML/data validation, active configuration contract checks, config-file classification, navigation schema validation, the Docker/local Jekyll build, Jekyll doctor, and compiled asset checks.

# Standard preflight; uses Docker Compose when the jekyll service is running,
# otherwise falls back to local bundle exec.
./scripts/validate

# Fast host-only checks used by the CI quality-checks job
./scripts/validate --quick

# Start Docker if needed and include optional test layers
./scripts/validate --full --start-docker

Test Suite¶

The theme includes a comprehensive test suite:

# Run all canonical tests
./scripts/bin/test

# Run specific test suites
./test/test_runner.sh --suites core,deployment

# Run with verbose output
./test/test_runner.sh --verbose

# Skip Docker tests
./test/test_runner.sh --skip-docker

Test Categories¶

Core Tests¶

Basic functionality and configuration:

./test/test_runner.sh --suites core

Tests include:

Configuration validation
Required files exist
Jekyll build succeeds
Layouts render correctly

Deployment Tests¶

Production readiness:

./test/test_runner.sh --suites deployment

Tests include:

GitHub Pages compatibility
Remote theme functionality
Build output validation

Quality Tests¶

Code quality and best practices:

./test/test_runner.sh --suites quality

Tests include:

Broken link detection
HTML validation
Accessibility checks

Frontend Playwright tests¶

Specs live in two sections under test/visual/, orthogonal to the execution tiers below:

core/ — cross-cutting quality/a11y/security/responsive baseline that applies regardless of feature (accessibility.spec.js, security.spec.js, styling.spec.js, responsive.spec.js, layout-chrome.spec.js, features-registry.spec.js).
features/ — one file per feature or tightly-scoped feature cluster, matching _data/features.yml's tests: links (search.spec.js, admin.spec.js, appearance.spec.js, navbar.spec.js, layouts.spec.js, …).

The Playwright runner is split into tiers selected via PLAYWRIGHT_PROJECT:

smoke (default) — every spec in core/ and features/ except the pixel-snapshot test.
snapshots — pixel screenshots of the homepage in each of the 9 theme skins, isolated in features/appearance-snapshot.spec.js (path-filtered in CI).
regression-{chromium,firefox,webkit} — all specs across all browsers (manual workflow_dispatch only).

# Smoke tier — starts Jekyll on 127.0.0.1:4000 unless BASE_URL is already set
./test/test_runner.sh --suites playwright
npm run test:smoke

# Pixel snapshots
./test/test_runner.sh --suites playwright_snapshots
npm run test:snapshots

# Reuse Docker Jekyll on :4000
BASE_URL=http://127.0.0.1:4000 ./test/test_playwright.sh

# Run just one section
npx playwright test --config=test/playwright.config.js --project=smoke test/visual/core/
npx playwright test --config=test/playwright.config.js --project=smoke test/visual/features/

# Refresh Linux snapshot baselines (uses Docker)
./test/update-snapshots.sh

Core tests also validate that a production Jekyll build emits main.css containing docs-layout rules (e.g. bd-layout).

Layout & viewport regression (`features/layouts.spec.js`, `core/responsive.spec.js`)¶

These specs exercise styling, layout containment, visibility, and advisory axe scans across five viewports defined in test/visual/fixtures.js:

Viewport	Size	What it guards
`mobile`	375×667	Offcanvas toggler, ToC FAB, landmark visibility
`tablet`	768×1024	Mobile quicklink chips, footer column balance
`midDesktop`	1140×720	Brand logo/title overlap (navbar container-query tier)
`desktop`	1280×720	Intro hero, code blocks, tables, docs chrome
`wideDesktop`	1320×720	Full nav labels without ellipsis

Canonical routes live in UI_ROUTES (home, quickstart, news section, features, theme preview, etc.). Tests call gotoOrSkip() so minimal fork installs skip missing pages instead of failing.

Third-party CSS/JS are bundled under assets/vendor/; see pages/_docs/development/vendor-assets.md (site: /docs/development/vendor-assets/) and npm run vendor:install to refresh.

Manual Testing¶

Local Preview¶

# Start development server
docker-compose up

# Test different pages
# - Homepage: http://localhost:4000/
# - Blog: http://localhost:4000/blog/
# - Docs: http://localhost:4000/docs/
# - Features: Enable mermaid/mathjax on test pages

Build Validation¶

# Check Jekyll configuration
bundle exec jekyll doctor

# Build with verbose output
bundle exec jekyll build --verbose --trace

# Validate HTML output
bundle exec htmlproofer _site --disable-external

Cross-Browser Testing¶

Test in multiple browsers:

Chrome
Firefox
Safari
Edge

Check:

Layout rendering
JavaScript functionality
Responsive design
Font rendering

Feature-Specific Testing¶

Mermaid Diagrams¶

Create a page with mermaid: true
Add diagram code
Verify rendering in browser
Check browser console for errors

./scripts/test-mermaid.sh

MathJax¶

Create a page with mathjax: true
Add math notation
Verify equations render
Check complex formulas

Comments (Giscus)¶

Note: Giscus requires production deployment to test.

Deploy to staging
Verify comments load
Test posting a comment
Check theme matching

Analytics (PostHog)¶

Set JEKYLL_ENV=production
Verify script loads
Check PostHog dashboard for events
Test DNT (Do Not Track) support

Continuous Integration¶

GitHub Actions runs tests on:

Pull requests
Pushes to main branch
Release tags

CI Configuration¶

.github/workflows/ contains:

Build validation
HTML proofer
Deploy workflows

Viewing CI Results¶

Go to repository → Actions
Click the workflow run
Review logs for failures

Writing Tests¶

Test Script Format¶

#!/bin/bash

test_something() {
    # Arrange
    local expected="value"

    # Act
    local result=$(some_command)

    # Assert
    if [[ "$result" == "$expected" ]]; then
        echo "[PASS] Test description"
        return 0
    else
        echo "[FAIL] Test description"
        echo "  Expected: $expected"
        echo "  Got: $result"
        return 1
    fi
}

Test Best Practices¶

Isolate tests — Each test should be independent
Clear assertions — Make expected vs actual clear
Descriptive names — Test names explain what's tested
Fast execution — Keep tests quick when possible
Document requirements — Note any prerequisites

Troubleshooting Tests¶

Common Failures¶

Error	Solution
Jekyll build failed	Check for syntax errors in Liquid/YAML
Missing file	Verify file exists and path is correct
Docker not found	Install Docker or use `--skip-docker`
Permission denied	Check file permissions

Debug Mode¶

# Enable debug output
DEBUG=1 ./test/test_runner.sh

# Run single test file
bash -x ./test/specific_test.sh

Local Setup — Development environment
Code Style — Coding conventions

User guide: Contributors who access testing via the public site can find an overview at Testing in the user-facing docs.

Testing¶

Preflight Validation¶

Test Suite¶

Test Categories¶

Core Tests¶

Deployment Tests¶

Quality Tests¶

Frontend Playwright tests¶

Layout & viewport regression (features/layouts.spec.js, core/responsive.spec.js)¶

Manual Testing¶

Local Preview¶

Build Validation¶

Cross-Browser Testing¶

Feature-Specific Testing¶

Mermaid Diagrams¶

MathJax¶

Comments (Giscus)¶

Analytics (PostHog)¶

Continuous Integration¶

CI Configuration¶

Viewing CI Results¶

Writing Tests¶

Test Script Format¶

Test Best Practices¶

Troubleshooting Tests¶

Common Failures¶

Debug Mode¶

Related¶

Layout & viewport regression (`features/layouts.spec.js`, `core/responsive.spec.js`)¶