Phase 4: close loot/refund leaks, landing-site garrisons, speed-scaled shipping, update/offsite tooling, broad e2e

Built:
- Economy: raid loot capped to the room the raiders have at home (rest stays with the victim, "lootLeft" in the report); trade escrow reserves its room so refunds always fit; trade accept refuses with no_room_for_goods when the goods would not fit. Loot destroyed per 30-day run 31-61k -> ~1k, refunds clipped ~27k -> 0 (docs/BALANCE.md, 3 seeds).
- End game: members of the holding alliance garrison landing sites (migration 005_site_garrison.sql), garrison counts in site defence, 72 h capture shield, withdraw route, POP view lists garrisoned units. Shipping capacity scales with world speed (3x world: winner 76-98 -> ~30-36 days after sites open). Bot runs show real contests (84-194 attacks on held sites at 1x).
- Ops: deploy/update.sh (backup, stop, ff-merge, npm ci --ignore-scripts, health check, rollback), deploy/upstream-offsite (rsync only), admin end-world snapshots results, EPIPE fix, docs/workflows/ filled in.
- e2e 62 -> 163 checks: units, raids/reports, alliance+messages, trade, end game, nb, PWA, reconnect, slow polling fallback (opt-in). Two app bugs found: message push ignored, "New" button stayed English.

Tested: npm test 274/274, openapi check, npm run e2e 163/163 (137 s), bot runs 50x30d seeds 1-3 and 140-day/60-day speed-3 end-game runs INVARIANTS OK, adversarial review pass (findings fixed or documented).
NOT tested: update.sh on a real box (systemd, real npm, runuser; stubs only, shellcheck not installed), garrison card/loot text on a phone, Node 22.13, humans. Known leaks left open and documented (BALANCE S6): loot/shipments can still overflow when stock arrives in flight, bounced backbone shipments lose overflow (reported to the sender). Garrison numbers (heldDefenceMult 2, shield 72 h) are first guesses from 2 seeds. Not claimed: balanced.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2MJaQ5LXAHzGnTDgceT14
This commit is contained in:
jvanvik 2026-10-12 00:58:12 +02:00
parent 9104480463
commit 3c23c51902
92 changed files with 4887 additions and 300 deletions

View file

@ -8,33 +8,35 @@ WebSocket push); stage 7 is partly done (bot harness + first balance pass, admin
playtested by humans, and has never run on the real target (Debian LXC + systemd + nginx Proxy Manager).**
Runnable and checked just now:
- `npm test`: 208 tests pass in about 7 s (rules as pure functions, commands/events/scheduler with a fake clock, REST over HTTP with every
- `npm test`: 264 tests pass in about 8 s (this tree) (rules as pure functions, commands/events/scheduler with a fake clock, REST over HTTP with every
response checked against its schema, push with a real `ws` client, client logic, bot harness, admin CLI). `npm run openapi` leaves
`docs/openapi.json` unchanged (it is in sync with the route table: 68 paths).
- `node tools/bot.js --players 50 --days 14 --speed 1`: about 3 s, `INVARIANTS OK`. `node tools/admin.js help` works.
- The deploy path from DEPLOY.md in a scratch directory, without systemd: first boot (migrations, world), `/healthz`, signup, a build,
`admin.js` status/worlds/users/inspect/events/replay/backup/verify-backup/ban/grant, SIGTERM + restart (state kept), SIGKILL + restart
after a build fell due (caught up in order), restore of a snapshot into a fresh `DB_PATH`.
- `npm run e2e` (`e2e/run.js`, Playwright): 61 checks pass (re-run after B4, 54 s).
- `npm run e2e` (`e2e/run.js` + `e2e/flows.js`, Playwright): 163 checks pass in about 137 s (smoke, units, movements + reports, reconnect, alliance + messages, trade, end game, nb, PWA); `E2E_SLOW=1` adds the 60 s polling-fallback suite. See "E2E" below.
Never run: real devices, Safari/iOS, a Proxmox LXC, a unit under systemd, the backup timer firing, nginx Proxy Manager (WebSocket upgrade,
idle timeouts, real client IPs), Let's Encrypt, a restore on the real box, load with many real sockets, any human player. Node 22.13 (the
stated minimum) was not run, only 24.21.
Known gaps, consolidated (the sections below have the detail and the reasoning):
- **Balance:** the first hour of a player who plays it all is limited by the 750 starting stock (about 12 builds), not by build time; a bigger stock was measured and not adopted (B4). nothing is tuned beyond B1, B2 and the B4 early-level build discount (BALANCE.md). Open: a 3x world's end game is as long as a 1x world's; after the opening days
nobody can contest a built-up landing site (no garrisons, E1); raid loot is split evenly and destroyed on overflow (S4); trade escrow
refunds are clipped at the Rack hall cap without a notice (S2); the capital runs out of things to build long before the backbone opens;
- **Balance:** the first hour of a player who plays it all is limited by the 750 starting stock (about 12 builds), not by build time; a bigger stock was measured and not adopted (B4). nothing is tuned beyond B1, B2 and the B4 early-level build discount (BALANCE.md). The end game follows world speed since B7 (3x: 29 to 35 days after the sites open, was 77 to 98) and a held landing site can be contested since E6 (garrisons, a
light static defence for held sites, a 72 h capture shield); both are first guesses measured with bots on two seeds (BALANCE.md B7 and E6), no human has played them, the static
defence outgrows what one POP can send around world age 280, and the bots do not react to `backbone_incoming`. Open: (S2 and S4, trade refunds clipped and loot/incumbent bonus destroyed on overflow, are fixed by B6: escrow keeps its room in the Rack hall, loot is limited to what home can bank; transport deliveries onto a full store, and a bounced backbone shipment that finds the hall refilled, still lose the overflow, reported with exact `cargoLost`; several raids from one home no longer share the same free room, S6; trade accept is refused with `no_room_for_goods` when the projected hall has no room); the capital runs out of things to build long before the backbone opens;
the bots are not humans.
- **Game rules not built:** cancelling training, sitters, night protection, a command to choose a new capital, power draw for units in
transit, alliance rename/description, trade order book / price history / counter-offers, multi-link trade deals.
- **Server:** no `backbone_open` push (sites are created lazily, E2); no `incoming` for confederates and no `town` hint for reinforcing
owners; token revocation reaches an open socket only at the next 30 s ping; tokens never expire and there is no change-password route; no APNs/FCM
sender; `reports.id` can be reused after a full cleanup (S3, low); no recovery sweeper if an `arrive`/`return` handler ever throws (R1);
HTTP connections per IP are not capped; admin `end-world` is a flag flip without ranking snapshot (DEPLOY.md section 6).
HTTP connections per IP are not capped.
- **Client:** no service worker (no offline mode), no light theme, no unit icons or battle simulator, no sound/system notifications; a few
one-tap destructive actions without confirmation; unread-badge state is partly in localStorage; ended-world locking is a DOM sweep.
- **Docs:** `docs/workflows/` (planned stage workflows, CLAUDE.md layout) is empty: no stage workflow was ever written there.
Found by the e2e and NOT fixed: after switching language live, screens that were already drawn keep ~15 English `aria-label`s (map zoom/label, refresh buttons, tab groups; visible
text is fine, a reload fixes it); an alliance message raises the More badge by 2 (the alerts feed and the unread message are both summed; by design of the badge, perhaps not intended).
- **Docs:** `docs/workflows/` has a short note per stage (written after the fact from NEXT and the git history; NEXT stays authoritative).
- **Not run in a browser:** takeover/capture flows, the confederation `alliedIncoming` view, real devices (see the client sections).
## Done
@ -117,8 +119,8 @@ a phone profile (markup in a message stays text, counter/over-limit, 429 countdo
move sheet, name-taken error, nb). Not run: real devices, Safari, Playwright suite, a long-list scroll test (>100 ranks), accepting an
invite when the alliance is full, alliances at the size cap.
Gaps: alliance rename/description and note editing have no UI (server has edit-note); no operator profile page; blocking needs an existing
conversation or message (the API takes an operator id and there is no name lookup); no live refresh (push is not wired, screens refetch
on open and on manual refresh); the alliance screen does not auto-refresh countdowns' targets after they pass; `App.state.operator.alliance`
conversation or message (the API takes an operator id and there is no name lookup); no live refresh (push is not wired for the alliance screen, which refetches
on open and on manual refresh; Messages follows the `message` push since the e2e pass); the alliance screen does not auto-refresh countdowns' targets after they pass; `App.state.operator.alliance`
is refreshed from `/me` after alliance changes but is stale if you are kicked while away until the screen is opened.
## Client: military, expansion, WebSocket
@ -197,7 +199,7 @@ Built (More group; strings in `strings/{trade,world,alerts}.js`, en + nb):
minute. Confederate Carol gets no push for `alliedIncoming` (known server gap): it shows when the Moves screen is opened or the poll runs, and never in her badge.
Found and fixed in the client: `report` pushes now also refetch movements (the incoming banner used to linger up to a few seconds after the arrival).
Not done / not claimed:
- Not run: real devices, Safari, a socket behind nginx Proxy Manager, throttled networks, the full Playwright suite (only the smoke e2e exists), long lists (offers > 25 per
- Not run: real devices, Safari, a socket behind nginx Proxy Manager, throttled networks, the combat/takeover/diplomacy flows in a browser, long lists (offers > 25 per
filter paged only up to what the sweep seeded).
- Ended-world locking is a DOM sweep, not per-button wiring: a new primary/danger button that is NOT a game command must carry `data-keep`. Messages, ranking and
settings screens are exempt by selector. Open send/trade sheets that were opened before the end stay usable until the next DOM change or a refused command.
@ -209,6 +211,41 @@ Not done / not claimed:
- Server bug found and fixed (tiny): `POST /reports/:id/share` threw a 500 when no `reportCleanup` event was pending (`ensureCleanup` needs `ctx.enqueue`);
`reportShare` now wraps its ctx with `ctxOf`, regression test `test/report-share-cleanup.test.js` (failed before).
## E2E (`e2e/`, run with `npm run e2e`)
`e2e/run.js` (smoke suites: desktop, phone, layout, push) and `e2e/flows.js` (feature flows) boot the real server in-process (temp SQLite, real clock, `WORLD_SPEED` 4000 or 1) and drive
headless Chromium through `playwright-core`; `e2e/lib.js` holds the shared plumbing (`bootServer`, `newPage` with console/HTTP error capture, `hurryMovement` / `hurryEvents`).
Default run: 163 checks, about 137 s, no console error or failed request tolerated; every page gets a default action timeout of 8 s (`lib.js`), the smoke suites included. `node e2e/run.js <suite...>` runs named suites; `slow` only by name or `E2E_SLOW=1`.
- **units**: train 2 script crews through the sheet (cost preview scales, queue row with count and countdown, count pill), the batch completes (event moved up), units appear in the home list and
the database, an unaffordable train is explained and disabled, hosted reinforcements are listed and "send home" removes them and starts the return.
- **movements**: scan and raid from the Moves screen send dialog (coordinates, type, unit picker, preview, send), both in the outgoing list with countdown and recall button; the defender's page gets
the banner and the Moves badge by push (type and units hidden), both arrive, report badges on both sides, the Reports list (unread, then read on open, server read flag), the raid and scan report
sheets, the defender-side report.
- **reconnect**: the page's socket is closed and new ones refused; a raid launched in the gap is not shown; on reconnect a NEW socket opens and the missed raid appears (refetch on ready); a second raid
then shows by push within 6 s with the socket open and no reload.
- **alliance**: found (button disabled until valid), invite by name, accept on the other page, a broadcast written in Messages > Alliance, the unread badge on the other page by push, the feed row
(unread, sender), mark read, a second broadcast appearing live.
- **trade**: post (escrow), the offer on the taker's Offers tab, Take with the POP picker, both legs in flight (moved up past the 10 minute minimum latency), goods swapped, `completed` on both
"mine" tabs, an alert.
- **endgame** (own world backdated to day 100.5): banner "appear in" + how-it-works before, "open" + 7 site cards after, the held site is `mine` with Manage, others show scan/attack, a non-member
sees no Manage; the Manage sheet, ship 300 fiber (toast, transport row marked as a site shipment, stock drops), it lands and the stockpile and ledger show it.
- **nb**: live switch after every screen was drawn in English (visible text), reload and fresh render (text, aria-label, placeholder, title), nav labels, dynamic keys (`err.*` in the train sheet,
`send.reason.*`), raw-key detector, English-leftover detector, and a check that no multi-word nb string is a verbatim copy of its English text. The same detector sweeps pages that hold real
data at the end of the movements, alliance, trade and endgame flows (reports and sheets, alliance tabs, messages tabs, trade tabs, site manage sheet and dialogs).
- **pwa**: manifest link, `application/manifest+json`, display standalone, start_url inside scope and serving the shell, theme colours equal to the meta tag, every icon 200 + image type + PNG
dimensions equal to the declared sizes, a maskable icon, apple-touch-icon, no service worker.
- **slow**: `WebSocket` removed from the page; the raid is not shown within 8 s, then shows by polling (59 s in the run, worst case about 60 s), no reload, socket never open.
How the checks were shown to be able to fail (each mutation applied, the named suite run, reverted): units queue never drawn (units), home list never drawn (units), reports never marked read
(movements), manifest served as octet-stream (pwa), no refetch on socket ready (reconnect), no reconnect scheduling (reconnect), alliance accept sends decline (alliance), trade accept with a wrong POP
(trade), open banner never drawn and ship endpoint misspelt (endgame), a copy-pasted English string in the nb table (nb), polling interval removed (slow); the nb detector additionally runs a planted raw
key, a planted English string and a deleted nb string on the live page, and a calibration run in English (it must see >= 8 English strings on the POP screen). Message push: the push check failed
on the tree before the fix below.
Real app bugs found: (1) `{type:'message'}` push was received and ignored (the unread badge and the open Messages list only moved on the 60 s poll or a tab switch): `messages.js` now refetches the unread
count and the visible inbox / alliance feed / open thread on that hint (regression: alliance flow, fails without it); (2) the "New" button of the Messages header stayed English after a live language
switch (built once in `render()`): now set in `onShow` (regression: nb flow, "switched live"). Not fixed, see the gaps above: stale aria-labels after a live switch; More badge double counts a message.
Not covered: takeover/capture, diplomacy moves, confederation `alliedIncoming`, rankings content, reports share to the alliance folder, settings devices list, Firefox/WebKit, real devices.
Caveats: flights and trade legs are moved up with `UPDATE movements/events` + `scheduler.arm()`, not by waiting; the end-game world is made 100.5 days old by moving `worlds.start_at` and
the 7 site rows come from `ensureSites` (what the first scan or attack does) with the holder written by SQL; the 60 s slow suite is not in the default run.
## Stage 6 (server half): WebSocket push
Built: `/ws` on the game's HTTP server (`server/push.js` transport, `server/pushHooks.js` routing), contract in `docs/PROTOCOL.md`.
Auth as first message (`ready` / close 4401), Origin check (`net.originAllowed`), per-IP cap `MAX_CONN_PER_IP` (HTTP 429), 1 KB frames,
@ -250,13 +287,15 @@ Not done / not claimed:
- **Nothing here is balanced.** Defence multiplier 12 (+3% per level), node cost 4000 x 1.045^(L-1) (about 7.2 M of each resource in total),
node time 2 h x 1.025^(L-1) (about 36 world days), 48 h offer ttl, 10 tiles of reach per exchange level, 500-per-side offers: first guesses,
no bot run when written (now there is one: BALANCE.md section 5 says an 18-member alliance needs 55 to 78 days after the sites open with 1000 per link, and does not finish in 175 days with 500).
- **No garrison at landing sites** (E1): defence is static, holders cannot station units. If the first world shows the race is decided by who
hits first, the next step is a small `site_units` table with reinforce-to-site and defenders in `landing.js`.
- **Garrisons at landing sites** (E6, migration 005 `site_stationed`, `server/military/garrison.js`, `POST /backbone/:id/withdraw`): built and tested (`test/endgame-garrison.test.js`,
`test/client-garrison.test.js`), bots garrison and assault (`tools/bot/endgame.js`, `--no-contest` turns it off). Not built / not claimed: no `captureLevelLoss`
(a taken site keeps its node level), no sitter, `tools/admin.js inspect-town` does not list garrison units, the site sheet was not opened in a real browser, a holder cannot send
a member's units away, confederates cannot garrison (members only).
- No `backbone_open` push at the moment the sites appear unless something touches a site first (E2); clients see the phase by polling status.
- Trade: no multi-link deals (each side <= 500), no price history or order book, no counter-offers, no notification when a new offer appears
in reach, offers are not hidden from enemies, the escrow of a taken-over POP goes to the conqueror.
- Ended worlds: movements still `out` and queued builds are left as they were (events cancelled, E3); `tools/admin.js end-world` only flips the status (no snapshot, no event cancellation) and `reset-world`
wipes a world's rows; neither runs the real `worldEnd` event (see DEPLOY.md section 6); several winners are impossible by design.
- Ended worlds: movements still `out` and queued builds are left as they were (events cancelled, E3); `tools/admin.js end-world` now runs the real `worldEnd` handler (snapshot, escrow back, events cancelled; ops part below); `reset-world`
wipes a world's rows without it (DEPLOY.md section 6); several winners are impossible by design.
- The ranking snapshot is the same numbers as `/rankings`, computed once; `/rankings/*` after the end still compute live from the frozen state.
- Client for all of this: see "Client: trade, end game, alerts" above.
@ -273,18 +312,25 @@ Server fix found by the harness: bounced shipments reported no overflow (`events
Tests: `test/bot-run.test.js` (12 players x 6 days twice, identical stats, invariants, the checks catch tampering, a fast world reaches the end game),
`test/bot-server-fixes.test.js`.
Not done / not claimed: **the game is not balanced**; the bots are not humans (no reinforcement, recall, messages, sitters; fixed sittings). Open
items from the pass: a 3x world's end game is as long as a 1x world's (shipping is per sitting; `capacityPerLink x speed` needs the client to stop
reading the constant); trade escrow refunds are clipped at the Rack hall cap silently; raid loot is split evenly over resources and destroyed on overflow;
the capital runs out of things to build ~50 days before the backbone opens at 4 sittings a day; a built-up landing site is out of reach of any
single-POP army (no garrisons, E1), so there is no fight for a site after the first days.
items from the pass: (a 3x world's end game being as long as a 1x world's is fixed by B7, `rules.transportCapacity`); (trade escrow refund clipping and loot destruction on overflow were fixed afterwards by B6, BALANCE.md section 9);
the capital runs out of things to build ~50 days before the backbone opens at 4 sittings a day; (a built-up landing site being out of reach of any single-POP army is
answered by E6: garrisons and waves, BALANCE.md section 5).
## Stage 7 (ops part): admin CLI, deploy files
Built: `tools/admin.js` (status, migrate, worlds, create-world, end-world, users, operators, ban/unban/revoke-tokens, inspect-operator/-town, events,
replay-events [--simulate], backup/verify-backup (VACUUM INTO, verified, pruned), reset-world, grant), `deploy/` (service, backup service + timer,
`upstream-admin` wrapper), `docs/DEPLOY.md`. Tests: `test/admin-cli.test.js`. Run by hand on 2026-10-11 in a scratch directory: see "Status at a glance".
Not done / not claimed: nothing of it has run under systemd or on Debian 13 (DEPLOY.md lists what was and was not run); no load test with real sockets;
world lifecycle through the CLI is partial (above); no scheduled "create next world".
Added later: `end-world` runs the real `worldEnd` event handler (final ranking snapshot so `/worlds/:id/results` is available, trade escrow back,
the world's other pending events cancelled, `world_ended` notifications; overdue events of that world are processed first; refuses without `--yes`),
quiet on EPIPE (`admin.js ... | head`); `deploy/update.sh` (git fast-forward update on the box: tools checked up front, service STOPPED before the tree changes (review R2), verified snapshot of the stopped db, `npm ci --ignore-scripts` only when the lock
changed, optional `--test` on a `git archive` copy as the service user, health check, rollback to the old rev, `--dry-run`); `deploy/upstream-offsite` + `.service`/`.timer` (off-box copy of the newest
verified snapshot to `OFFSITE_DEST`, skipped unless configured). Tests: `test/admin-endworld.test.js`, `test/deploy-update.test.js` (stubbed systemctl/curl/npm),
`test/deploy-offsite.test.js` (local directory as destination; needs rsync; scp was removed in R2).
Not done / not claimed: nothing of it has run under systemd or on Debian 13 (DEPLOY.md lists what was and was not run); `update.sh` and the off-site
copy were run only against stubs / a local directory (no real ssh, rrsync, systemd, runuser, npm ci or /healthz; shellcheck and dash were not available); a migration cannot be rolled back by
`update.sh` (it prints the snapshot to restore); `end-world` cannot repair a world ended by the old flag-flip version (no snapshot); the notification
rows the CLI writes are not pushed by a running server; no load test with real sockets; no scheduled "create next world".
## Not done
- Stage 7 remainder: balance passes beyond B1, B2 and B4 with the open items above, a real world lifecycle (start, natural end, reset) driven from the CLI,
load test over HTTP/WebSocket, the full e2e suite and a human playtest. Everything else in CLAUDE.md "Working in stages" is built.
load test over HTTP/WebSocket, e2e for takeover/capture/diplomacy/confederation views, and a human playtest. Everything else in CLAUDE.md "Working in stages" is built.