Benchmark in real Chromium too, and only call a row slower when the samples say so

npm run perf -- --browser times the same workloads in Chromium as a desktop
and as a phone (390x844 at 3x, CPU slowed 4x), one browser per version. Each
run collects garbage first and ends by forcing layout and the map canvas, so
deferred work counts. The jsdom DOM is about 30x slower than a browser's and
has no layout, so the interface rows need this.

Shared machines swing single runs by 30% on identical code, CPU time
included. A row now counts as slower or faster only when it moved 15% and a
Mann-Whitney U test gives p < 0.01 (15 runs a side by default). Identical
code: no flags in three full runs; a routing slowdown planted in a copy is
flagged in jsdom and on the phone.

The panel workload renders both selections every run instead of alternating,
which split its samples in two. CI drives the runner's Chrome through
playwright-core, no browser download.
This commit is contained in:
Frederic Jung 2026-10-26 12:04:57 +01:00
parent c88bad3eef
commit c76dd4d556
No known key found for this signature in database
11 changed files with 236 additions and 42 deletions

View file

@ -38,7 +38,7 @@ Other files:
| `site/site.webmanifest`, icons | PWA install metadata |
| `scripts/build.sh` | Copies `site/` to `_site/`, stamps `__VERSION__`/`__BUILD_DATE__`, bakes release notes, writes `version.json` |
| `scripts/release-notes.js` | Extracts a version's CHANGELOG bullets |
| `scripts/perf-bench.js`, `scripts/perf-fixture.js` | Big-network benchmark (base vs head, posted on every PR by `perf.yml` + `perf-comment.yml`) and the generator of its save, `tests/fixtures/big-network.json` |
| `scripts/perf-bench.js`, `scripts/perf-browser.js`, `scripts/perf-stats.js`, `scripts/perf-fixture.js` | Big-network benchmark, base vs head, headless and in Chromium as a desktop and a throttled phone, with a significance test per row (posted on every PR by `perf.yml` + `perf-comment.yml`), and the generator of its save, `tests/fixtures/big-network.json` |
| `scripts/i18n-check.js` | Compares every `tr`/`trn`/`N_` literal and `data-i18n` text with each dictionary |
| `tests/harness.js` | Loads the page in jsdom with a seeded `Math.random` and a fake canvas |
| `.github/workflows/ci.yml` | Tests and build on every push and PR |