J3ZZ

Testing

This document explains the automated testing system for the website to prevent breaking changes.

⚠️ CRITICAL: All testing happens LOCALLY. GitHub Actions does NOT run tests - it only builds and deploys!

Looking for a step-by-step tutorial? See Testing & Deployment Tutorial for practical, hands-on guidance with examples.

Overview

The site uses a comprehensive testing suite that runs on your local machine before pushing. Your Lefthook git hooks automatically enforce these tests:

yamllint validates:

html-proofer validates:

Lighthouse CI validates:

Running Tests Locally

YAML Linting

yamllint .

This is very fast and checks:

Note: Automatically ignores node_modules/, vendor/, _site/, and .bundle/ directories.

bundle exec rake test

This is fast and checks:

bundle exec rake test_external

This is slower but also checks:

Lighthouse CI Tests

npm run lighthouse

Note: Requires Chrome/Chromium to be installed. In WSL environments, this may not work locally. Install Chrome dependencies if you need to run these tests locally.

Tests:

npm run test:print

Validates print layouts and generates PDFs for manual review. Tests:

Output: PDFs saved to ./print-test-results/ directory for manual inspection.

Note: Requires Chrome/Chromium with proper dependencies. In WSL or headless environments where Chrome is not available, print tests will automatically skip locally. Install Chrome dependencies if you need to run these tests locally.

Pages tested:

All Tests Together

npm run test:all

Runs html-proofer, Lighthouse CI, and print tests sequentially.

Other Commands

# Just build the site
bundle exec rake build

# Clean the build directory
bundle exec rake clean

# Default task (runs test)
bundle exec rake

Automated Testing with Lefthook

⚠️ IMPORTANT: This project uses Lefthook git hooks to enforce ALL tests locally. GitHub Actions does NOT run tests.

Git Hooks (Automatic Enforcement)

Pre-commit hook (runs when you git commit):

Pre-push hook (runs when you git push):

How It Works

  1. Make your changes
  2. Run git commit - pre-commit hook validates YAML
  3. Run git push - pre-push hook runs ALL tests automatically
  4. If tests pass → push succeeds → GitHub deploys
  5. If tests fail → push is blocked → fix errors and try again

Skip hooks (use with extreme caution):

LEFTHOOK=0 git push
# or
git push --no-verify

⚠️ WARNING: Skipping hooks bypasses ALL testing. Only skip if you’ve already run ./test-before-push.sh manually and all tests passed.

GitHub Actions Workflow

.github/workflows/jekyll.yml - Build and deployment ONLY:

That’s it! No tests run in GitHub Actions. All quality gates are enforced locally via Lefthook.

Viewing Local Test Results

Lighthouse Reports:

Print Test PDFs:

Common Issues and Solutions

yamllint Issues

YAML Syntax Errors

syntax error: expected <block end>, but found '<block mapping start>'

Solution: Check indentation - YAML requires consistent 2-space indentation. Ensure no tabs are used.

Line Too Long

line too long (150 > 120 characters)

Solution:

Wrong Indentation

wrong indentation: expected 4 but found 2

Solution: Ensure consistent 2-space indentation. Check parent element indentation level.

Trailing Spaces

trailing spaces

Solution: Remove whitespace at end of lines. Most editors can do this automatically on save.

html-proofer Issues

Images without src/srcset

image has no src or srcset attribute

Solution: Ensure all <img> tags have a src attribute or use srcset for responsive images.

internally linking to /works/foo/, which does not exist

Solution:

Missing files

internally linking to /assets/cv.pdf, which does not exist

Solution:

Lighthouse CI Issues

Low Performance Score

Performance: 72/100 (required: 85/100)

Common causes:

Solutions:

Accessibility Violations

Categories:accessibility: 78/100 (required: 85/100)

Common causes:

Solutions:

SEO Issues

Document does not have a meta description

Solution:

Color Contrast

Background and foreground colors do not have sufficient contrast ratio

Solution:

QR Codes Not Visible

QR codes found but none are valid/visible in print mode

Solution:

Print-only elements not visible

Solution:

No-print Elements Still Visible

No-print elements still visible

Solution:

Configuring Tests

yamllint Configuration

Edit .yamllint to customize YAML linting behavior:

---
extends: default

ignore: |
  node_modules/
  vendor/
  _site/
  .bundle/

rules:
  line-length:
    max: 150              # Maximum line length
    level: warning        # Can be: warning, error, or disable

  indentation:
    spaces: 2             # Number of spaces for indentation
    indent-sequences: true

  truthy:
    allowed-values: ['true', 'false', 'yes', 'no', 'on', 'off']

  document-start: disable # Don't require --- at start

Key settings:

html-proofer Configuration

Edit Rakefile to customize test behavior:

options = {
  :disable_external => true,     # Set to false to check external links
  :ignore_missing_alt => true,   # Set to false to enforce alt tags
  :check_html => true,            # Validate HTML structure
  :enforce_https => false,        # Set to true to require HTTPS
  # Add URLs to ignore:
  :ignore_urls => [
    /localhost/,
    /example\.com/
  ]
}

Lighthouse CI Configuration

Edit lighthouserc.json to customize Lighthouse behavior:

{
  "ci": {
    "collect": {
      "numberOfRuns": 3,
      "url": [
        "http://localhost/index.html",
        "http://localhost/bio/index.html"
      ]
    },
    "assert": {
      "assertions": {
        "categories:performance": ["warn", {"minScore": 0.85}],
        "categories:accessibility": ["error", {"minScore": 0.85}],
        "categories:seo": ["warn", {"minScore": 0.85}]
      }
    }
  }
}

Key settings:

Edit test-print.js to customize print testing:

const CONFIG = {
  siteDir: './_site',           // Built site directory
  outputDir: './print-test-results',  // PDF output directory
  pages: [
    {
      path: 'index.html',
      name: 'homepage',
      checks: ['qr-code', 'print-layout']
    },
    // Add more pages to test
  ],
  a4: {
    width: 210,    // mm
    height: 297,   // mm
    margin: 15     // mm
  }
};

Available checks:

Adding new pages:

{
  path: 'works/new-work/index.html',
  name: 'new-work',
  checks: ['qr-code', 'print-layout', 'metadata']
}

Best Practices

  1. Let Lefthook handle testing automatically (recommended)
    • Just commit and push normally
    • Pre-commit hook: Quick YAML check on staged files
    • Pre-push hook: Complete test suite via ./test-before-push.sh (automatic)
    • No need to run ./test-before-push.sh manually
  2. Run tests manually during development (optional) 
    ./test-before-push.sh

    Run this while developing to catch issues early, before committing. The pre-push hook will run the same tests again automatically when you push (but will pass quickly since you already fixed everything).

  3. Quick YAML check during editing 
    yamllint .

    Fast check for YAML syntax errors in config files and front matter.

  4. Fix issues immediately - Don’t accumulate broken links or test failures

  5. Check external links periodically 
    bundle exec rake test_external
  6. Test print layouts after CSS changes 
    npm run test:print

    Review generated PDFs in ./print-test-results/ to ensure print styles work correctly.

  7. Run full Lighthouse tests before major releases 
    ./test-before-push.sh --full

    Includes performance, accessibility, and SEO validation.

  8. Never skip hooks unless absolutely necessary
    • Skipping hooks = deploying untested code
    • Only skip if you’ve already run ./test-before-push.sh successfully
  9. Template/Example works - May intentionally have broken links for demonstration purposes. Add them to ignore_urls in Rakefile if needed.

Ignoring Specific Issues

To ignore specific files or URLs, edit Rakefile:

# Ignore specific URLs
:ignore_urls => [
  %r{/works/template-example/}  # Ignore template works
]

# Ignore specific files
:ignore_files => [
  /templates\//
]

Additional Testing Options (Future)

If you need even more comprehensive testing, consider:

Current testing suite (yamllint + html-proofer + Lighthouse CI + print tests) enforced locally covers:

All critical tests run automatically via Lefthook hooks before you can push!

This is comprehensive for most portfolio websites!