My Click Test Passed 11 of 12 Broken Mobile Menus.

I built a click checker to catch a broken mobile menu. Then I ran it against twelve preserved broken copies. Eleven passed.

My mistake cost me a reliable release gate: eleven known-broken copies received a pass.

The remaining copy failed too, but for the wrong reason. Its link targets had zero height. The check that claimed the menu had opened still passed.

This wasn’t a new browser regression. These are measurements recorded on August 26, 2026, while reviewing an AI-assisted static-site pipeline. I’m publishing the archived result now because the failure was in my definition of success.

The screenshots couldn’t show a click that never happened

My generated pages used a hamburger button on narrow screens. The script added is-open; the stylesheet expected .active. Clicking changed a class name that the relevant CSS didn’t use.

I had screenshots and a structural QA gate. Neither checked whether the menu actually opened after interaction.

For one broken/fixed pair, the recorded screenshots were both 560 × 5200 pixels and 378,552 bytes. Their SHA-256 hashes matched. The page was captured before clicking the button, so both menus were closed.

I’d treated a picture of the starting state as evidence about the next state. The pictures weren’t lying. I was asking them a question they couldn’t answer.

Thirty-six checks, zero tests of the missing behavior

The structural gate reported 36 checks and zero failures on a broken copy. It checked useful things: missing files, layout ingredients, and basic structure. It didn’t exercise menu opening.

Its output even included a manual checklist for mobile touch targets and hamburger behavior. I missed that qualification when I read the successful summary.

My response was to add a browser checker using the Chrome DevTools Protocol. It called the button’s DOM click() method, inspected panel coordinates, and followed navigation links. That sounded much closer to real use.

It wasn’t close enough.

Archived test result Broken copies
Passed all eight checks 11 of 12
Failed a touch-target check 1 of 12
Changed aria-expanded after clicking 12 of 12
Changed panel opacity after clicking 0 of 12
Changed panel visibility after clicking 0 of 12
Changed panel top or height after clicking 0 of 12

The twelve copies were related outputs from one pipeline. This isn’t an estimate of failure rates across AI-generated websites. It’s a regression set that my own checker should have rejected.

Coordinates weren’t visibility

Seven copies still had hidden panels after the click. Those panels nevertheless had measurable bounding boxes.

Preserved example Opacity Visibility Panel height
A, one-page version 0 hidden 265.4 px
C, ten-page version 0 hidden 809.9 px
D, ten-page version 0 hidden 735.9 px

My check asked whether the panel was near the header. A hidden panel can have a perfectly reasonable position. I’d named a position check as though it were a visibility check.

MDN’s visibility reference explains the distinction: hidden elements can still occupy layout space. A nonzero rectangle wasn’t proof that a reader could see the menu.

The checker also called click() directly on navigation links and confirmed their destinations. Those calls worked even when the links weren’t reachable through the visible page.

The separate measurement used document.elementFromPoint at the first link’s center. In the seven hidden-panel cases, the hit landed on a hero heading instead of the link. That’s where a user’s pointer would have gone.

The HTMLElement.click reference describes programmatic activation. It isn’t a promise that the element is visible, unobstructed, or usable by touch.

Likewise, aria-expanded=true only showed that my handler changed an attribute. It didn’t demonstrate agreement between the script, CSS, and rendered panel. All twelve broken copies changed that attribute successfully.

The regression test I actually needed

I now define the acceptance question before choosing the browser operation: can a person open the menu and activate a visible link?

For this case, a useful review sequence is:

  1. Record the actual viewport width, rather than trusting the requested window size.
  2. Activate the hamburger through pointer or touch input.
  3. Inspect the rendered panel after any transition has settled.
  4. Check visibility, opacity, dimensions, and clipping together.
  5. Hit-test a link point, allowing for a child element inside the link.
  6. Activate that point and verify the expected destination.

That’s a stronger acceptance plan, not a claim that I’ve completed a universal replacement checker. Keyboard operation and assistive-technology behavior need their own checks too. I don’t get to infer those from a pointer test.

Don’t count a test crash as a detected product bug

The archived rerun had another trap: rapid launches initially produced port-conflict errors. Those weren’t evidence that the checker detected broken menus. I had to separate harness failures from actual assertion results before counting passes.

I also don’t combine the eleven passes with the one failure into a comforting detection rate. That one failure concerned touch-target size; the menu-open assertion was still wrong.

The related extension testing story was about a person finding problems I hadn’t encountered. This one is about an automated check confidently answering a different question.

I’d have kept trusting the checker if I hadn’t tested it against known-broken copies. The useful regression suite wasn’t a collection of green examples. It was the set of failures the checker had to recognize.

Similar Posts