GitHub Made Pull Request Stacks Visible. Now Test Every Layer

A green pull request stack is not the same thing as a correct interface.

GitHub recently made stacked pull requests much easier to understand. A stack indicator now shows where the current PR sits in the chain, and the stack map lets reviewers move between layers. The gh stack workflow also handles much of the branch and rebase work that used to make stacks painful.

That is a real improvement. A PR targeting another feature branch no longer looks like an oddly based change with missing context.

But we saw the limit of that improvement in a two-PR stack. The first layer supplied the foundation. The second added the frontend. GitHub recognized the chain and showed layer 2 of 2. The final interface still contained a visual regression.

Nothing was wrong with the branch graph. The pixels were still wrong.

The lesson is simple: stack-aware coordination needs stack-aware verification. We should test the visual result of each layer, then test the assembled stack before it lands.

That is the workflow we are building around ShakaPerf.

The stack map cannot judge the rendered page

A pull request stack is an ordered chain. The bottom PR targets trunk, usually main. Each PR above it targets the branch below it. GitHub can show that structure, enforce protection rules, rebase the chain, and merge its layers in order.

Those guarantees answer useful questions:

• Which PR depends on which?
• Is every layer approved?
• Did CI run?
• Is the stack ready to merge?

They do not tell us whether a shared CSS rule moved a control, a new data shape broke a layout, or the phone view now wraps incorrectly.

A reviewer may notice those problems in code. We should not depend on that. Small frontend changes can have effects far outside the lines in the diff.

The stack map gives us structural context. Screenshot comparison gives us rendered evidence. We need both.

A stack needs two visual checks

For a normal PR, the A/B comparison is straightforward: the base branch is the control and the PR branch is the experiment.

A stack needs two comparisons:

• Layer check: compare the current PR with its direct base. This tells us whether that PR introduced the visual change.
• Stack check: compare the top branch with trunk. This tells us whether the completed feature is correct.

Adjacent comparisons keep each report focused on the layer that introduced the change.

If every layer is compared only with trunk, changes from lower PRs keep reappearing in upper reports. The evidence becomes noisy, and it is harder to see where a regression entered.

If we compare only adjacent layers, each report stays focused. But we never explicitly check the final product assembled from the entire chain.

The layer check tells us where a difference entered. The stack check tells us whether the finished composition is acceptable. That is exactly what we want from the code review: focused changes locally and coherent behavior globally.

One scenario can produce all the evidence

ShakaPerf runs a control application and an experiment application side by side in production mode. A single Playwright scenario drives both.

For a layer check, control points at the PR's direct base and experiment at its head. For the final stack check, control points at trunk and experiment at the top branch.

The final comparison verifies the interface assembled from the entire stack.

The scenario can stay small:

import { abTest } from "shaka-shared"

abTest(
  "Compact menu editor",
  {
    startingPath: "/admin/menu",
    visregSelectors: ["viewport"],
  },
  async ({ page }) => {
    await page.getByRole("button", { name: "Compact sections" }).click()
    await page.getByTestId("compact-section-list").waitFor()
  }
)

After the interaction finishes, ShakaPerf captures the same state from both applications and compares the images. The same abTest() definition can also drive performance and accessibility checks. We do not need separate scripts to reach the same page state.

The local visual loop is short:

yarn shaka-perf servers
yarn shaka-perf compare --categories visreg

The result is a self-contained HTML report with control, experiment, and highlighted differences. The reviewer sees the rendered result without reproducing the environment or guessing what a CSS change will do.

A difference is evidence, not automatically a defect

Some PRs are supposed to change the interface. A visual diff should not reject every new button or adjusted margin.

It should make the change explicit.

The useful question is not, "Did any pixel move?" It is, "Are these the pixels we intended to move?" Showing the control, experiment, and difference makes that answer quick.

This is especially useful in a stack. A backend PR may have no visible effect by itself. The frontend PR above it can show the exact rendered consequence. The full-stack report then confirms the completed flow.

Reliable tests matter. Dynamic timestamps, unfinished animation, late fonts, and inconsistent browser state create noise. We wait for real ready conditions, seed the same data, and capture the same selectors at named viewports. Retries and pixel thresholds are useful, but they should not hide a repeatable layout shift.

Rerunning the same commits should produce the same conclusion. If it does not, stabilize the scenario before making it a required check.

Rebasing must invalidate old visual proof

Changing a lower PR changes every branch above it. GitHub can rebase the stack and rerun CI, but an earlier visual report no longer proves anything about the new chain.

Every report should belong to exact control and experiment commits. After a lower layer changes:

1. Rebase the upper layers.
2. Rerun the affected layer comparisons.
3. Rerun the trunk-to-top comparison.
4. Require the new reports before merge.

This is stronger than pasting a screenshot into a PR comment. The status follows the commit, so stale evidence cannot remain green after the stack changes.

The workflow we would use now

The practical version is small:

1. Split the work into focused branches, with foundations below and rendered behavior above.
2. Create the stack with GitHub's UI or gh stack.
3. Compare every PR with its direct base.
4. Compare the top branch with trunk.
5. Publish the visual report and block unexpected changes.
6. Rerun both checks after a rebase.
7. Merge only when every layer and the complete stack tell the same story.

This adds CI work, but it also makes that work focused. A layer can run only the scenarios related to its surface area. The top-of-stack check can cover the broader critical paths.

GitHub's stack button is welcome because it makes dependency visible. The next step is making rendered correctness just as visible.

A stack is a graph of changes. Users never see the graph. They see the final page.

Aloha, Roman