Performance benchmark on a big network, posted on every PR

scripts/perf-bench.js times routing, a game day, an action, panels, the
NOC dashboard and the map on tests/fixtures/big-network.json (104 POPs,
950 PNIs, 832 caches), base and head in alternating turns, and says
whether routing results changed. perf.yml runs it on every PR against
the base; perf-comment.yml posts the table as one updated comment.
New performance skill, CLAUDE.md rule, PR template and docs.
This commit is contained in:
Frederic Jung 2026-10-26 06:39:23 +01:00
parent ad8b1bebcb
commit 308c3f7045
No known key found for this signature in database
13 changed files with 322 additions and 1 deletions

View file

@ -0,0 +1,61 @@
---
name: performance
description: Measure and demonstrate the performance of every Series of Tubes change on a very large network, compare it with main, and report the numbers.
---
# Performance is always demonstrated
Every change to the game (anything in `site/`) shows its performance on a very large network,
compared with `main`, before it is merged or released. A speed claim without numbers does not
count, and neither does "it feels fine on my small game". Most players who complain about lag
have huge late-game networks, and most of them are on phones.
## Measure
```bash
git show origin/main:site/index.html > /tmp/base.html
npm run perf -- --base /tmp/base.html # about a minute
```
`scripts/perf-bench.js` loads `tests/fixtures/big-network.json` (about 100 POPs, 150 links,
950 PNIs, 830 cache boxes, 110 IX ports, 170 customers) into the base and into this checkout, in
one process, and times them in alternating turns. It prints a Markdown table (median of 7 runs):
- Routing: `compute()`
- One game day: `dayTick()`
- An action: `after()`
- Side panel and left list
- NOC dashboard tabs
- Map at three zoom levels
and says whether both versions route the save's traffic identically.
Without `--base` it just times this checkout. `--runs N`, `--md FILE`, `--json FILE` and
`--summary FILE` are there for CI and for digging.
## Demonstrate
- **Every pull request**: the Performance workflow runs the same comparison and posts the table as a
comment on the PR (updated on every push). Read it before merging. Locally, paste the table into
the PR description or the commit summary you report.
- **Everything you hand back to Frederic** that touched `site/`: include the table, or at least the
rows that moved, in your summary. Say plainly when something got slower.
- **Before a release**: compare the release against the previous tag
(`git show v<last>:site/index.html > /tmp/base.html`) and mention notable changes.
## Read the table
- Below about 2 ms, differences are noise. The verdicts already ignore them.
- ⚠️ slower (15% or more) or 🔴 slower (50% or more) needs a reason in the PR, or a fix. A new
feature may cost time; a refactor or a fix should not.
- "Routing results differ" is expected when a change is meant to change gameplay (new cities,
balance, routing rules), and a bug when it only promises to be faster.
- It is headless (jsdom with a recording canvas): it measures our JavaScript, not the browser's
painting. For drawing changes, also look at the game on a phone.
## Keeping the fixture useful
The save loads through `migrate()` like any player's save, so new state gets its defaults. Only
regenerate it (`node scripts/perf-fixture.js`, several minutes) when the save format changes in a
way old saves cannot load, or the game grows new heavy systems the save does not exercise. Say in
the commit that the baseline changed, because numbers before and after are no longer comparable.