levikaldor.hu quality gate

One command that checks every page before a release, and stops the push if anything got worse.

type
Test automation
my role
Design, development, testing
technology
Astro, TypeScript, Playwright, axe-core, Lighthouse, Vitest

Starting point

Website quality decays quietly. A new image slows loading, a new colour breaks contrast, an embedded widget sends data to a third party. Checking all of this by hand after every change is not realistic, so I gave the job to a machine.

What it checks

One command (npm run audit) builds the site, goes through every page, and fails if any rule is broken:

  • SEO: title and description length and uniqueness, canonical, hreflang pairs, a single H1, structured data, sitemap, robots.txt, broken internal links.
  • Accessibility: axe-core with the WCAG 2.2 AA rules, in light and dark theme, on mobile and desktop sizes. Keyboard use is tested with Playwright too: skip link, visible focus, the mobile menu and Escape, the form error summary.
  • Reflow: no page scrolls sideways at a width of 320 px.
  • Privacy: no browser request may go to a third party, no cookie may be set, and every external service must be listed in the data processing register.
  • Legal pages: they exist in both languages, are linked from the footer of every page, and show a current effective date.
  • Old addresses: no previously public URL may return a 404.
  • Lighthouse: 100 in all four categories, on mobile and desktop, on one page per template.

Decisions

A git hook, not CI. The check runs in the pre-push hook, and only for pushes to the main branch, because Vercel deploys from there. If no watched file changed since the last full run, it is skipped, so a second push does not wait another five minutes.

Lighthouse on a sample. The first version ran Lighthouse on every page and took more than an hour. Now it measures one page per template, and PageSpeed Insights measures every page after deployment.

The result on the site. After a full passing run the audit writes the date and the number of pages and tests to a small JSON file. The home page and the footer show the last check from it.

How I tested it

The checks are tested themselves with Vitest, test-driven: every rule has a test that failed first and passed only after the fix. An automated code review found bugs in it as well. One of them was that an error page was silently checked in place of another. A failing test showed that first, then I fixed it.

What I would do differently

Lighthouse is sensitive to machine load: with a development server running next to it, flawless code can score 98 or 99. Today I would handle measurement noise from the start and rerun a measurement once automatically after a technical failure. The second part has since been added.