Upstream/docs/DECISIONS.md
jvanvik d84b0c9d12 Fix review findings: proto smuggling, trade-leg recall, escrow cap, static headers, rate limits, rankings DoS
Fixed: validator rejects __proto__/constructor keys at any depth (transport capacity bypass); trade and site shipments
cannot be recalled; trade escrow refund capped by Rack hall; CSP/nosniff on static files and connect-src 'self';
IPv6 /64 limiter keys + LRU eviction + per-username login bucket + scrypt concurrency cap; 5 s footprint cache for
rankings; rename uses text.cleanText; 429s no longer pinned in the idempotency store; 30-day notification prune;
units_offline notification on first offline. Accepted items listed in docs/DECISIONS.md R1.
Tested: npm test 182/182 incl. test/review-fixes.test.js (regression tests). Not re-reviewed after the fixes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2MJaQ5LXAHzGnTDgceT14
2026-10-11 19:06:51 +02:00

232 lines
20 KiB
Markdown

# DECISIONS
Short log of non-obvious decisions and why. Newest at the bottom.
## D1 — `node:sqlite` as the driver
Dev machine runs Node 24; the deploy target (Debian 13 LXC) ships Node 22+. `node:sqlite` is used through `server/db.js` only, so
`better-sqlite3` is a one-file swap if it ever misbehaves. Parameters are always positional.
## D2 — Scheduler processes ALL due events before any command or read
`scheduler.runDue()` runs every pending event with `due_at <= now` globally, in `(due_at, id)` order, not just the ones touching the
town in question. It is simpler and makes fairness trivial (nothing resolves out of order). Cost is nil in practice because the timer
processes events within milliseconds of their due time; it only matters after a restart (catch-up).
## D3 — One `towns.res_at` for all lazily-grown values
Fiber, hardware, capital, power battery, footprint and loyalty are all settled together at `res_at` (`state.settle`). Any change that alters a
rate (building finished, unit trained or lost) settles first. `rules.resourcesAt` is therefore exact between settles.
## D4 — Power battery uses the same lazy model
Battery = (amount, rate = net power x world speed, cap = Battery bank). Units are offline while battery = 0 and net < 0
(`rules.unitsOnline`). The 24 h decommission countdown is an event scheduled from `rules.batteryEmptyAt`.
## D5 — Rural areas are held by proximity in v1
DESIGN.md leaves "contested or just held" open. v1: a POP within `ruralReach` (2) tiles of a rural tile gets its +25% (max 3, nearest first).
Computed from the seed, no state. Revisit after the first world.
## D6 — First world is 101 x 101, not 301 x 301
DESIGN.md's own open question. `mapRadius` is world data (`worlds.map_radius`), default 50.
## D7 — `docs/openapi.json` is generated
`tools/gen-openapi.js` builds it from the route table (schemas live in route meta + `server/routes/schemas.js`). `npm test` fails when the
committed file is stale, which satisfies "updated in the same commit as any route change" mechanically.
## D8 — Resource fields are slot-keyed
Buildings table key: `f1..f18` for resource fields (type fixed at founding, 4 per type + 2 wild), the building type for infrastructure.
Build "slots": 1 concurrent build, 2 from Core router 10 (queue = pending `buildDone` events).
## S5-1 — Alliance size cap is stored, only ever raised
DESIGN says the Peering office level sets the size you can found. The cap is stored at founding (`10 + 2 * best level`). A lost or
downgraded building must never kick anyone, so the cap is monotonic: commands that depend on it (invite, accept, transfer) raise it if
the CURRENT leader's best office would give more; reads show `max(stored, formula)` without writing. The floor is therefore 10 even
for a leader without an office after a transfer. The office requirement (>= 1) is checked when founding and when accepting an invite,
not afterwards.
## S5-2 — Diplomacy move table is data, three kinds of change
`SOCIAL.diplomacyMoves` (data.js) says for each (from, to) whether it is `immediate`, a `proposal` or `delayed`. War from neutral is
immediate (no consent needed). Anything that leaves a pact (nap/confed -> neutral/war, confed -> nap) is `delayed`: 24 h / world speed
via a `diploChange` event, so you cannot betray a partner without warning. Pacts and peace need acceptance. While a delayed change is
pending the pair is locked (`diplomacy_pending`); only the starter can revoke. A pair at `neutral` with nothing pending has no row in
`alliance_relations`. Acceptance re-validates against the current state, because the relation may have moved since the proposal.
`war -> confed` is not allowed (go through a pact first).
## S5-3 — Membership lives in two tables, one writer
`operators.alliance_id` (from 001, cheap joins in /map, /me) and `alliance_members` (role, joined_at, one leader per alliance by a partial
unique index). `alliances.js` addMember/removeMember are the only writers. Alliance rows are hard-deleted on disband; messages keep a
`alliance_tag` snapshot and carry no foreign key to alliances so broadcasts survive.
## S5-4 — Last online comes from tokens
No new column: `lastOnline` = newest `tokens.last_used_at` of the member's user, which auth.js refreshes at most once a minute. So it
is accurate to about a minute and includes revoked tokens (a logout still counts as having been online).
## S5-5 — Message rate limit is DB-backed, not the in-memory Limiter
`server/http.js` Limiter keys on monotonic real time and forgets on restart; message limits are a product rule that should follow the
(fake) clock and survive restarts, so the sender's recent rows in `messages` are counted (`SOCIAL.messageLimits`, indexed by
`messages_sender`). The per-token/IP limiters still apply on top.
## S5-6 — Messages: per-recipient rows, sanitising on write
One `messages` row plus one `message_recipients` row per recipient (broadcast fan-out at send time: members who join later do not see
old broadcasts; members who leave keep what they received). Read/delete state is per recipient; the sender's copy is hidden with
`sender_deleted_at`; the row is purged when nobody has a copy. A blocked sender is silently not delivered to (no row), so blocking does
not leak. Text is sanitised once on write (`server/text.js`): C0/C1 controls, bidi overrides and isolates, zero-width and invisible
tag characters are stripped, NFC, `\r\n` -> `\n`; ZWJ/ZWNJ are kept for emoji. Clients render with `textContent` only.
## S5-7 — Rankings are computed per request, no cache
Footprint for a whole world comes from two queries (towns, buildings) and `rules.footprintAt`; fine for the expected 50-300 operators.
If it ever shows up in a profile, cache per (world, minute). Cursor is a rank offset because rankings are a computed snapshot, not rows.
## S5-8 — Route params and list order
`inviteId`, `noteId`, `proposalId`, `otherId`, `operatorId` were added to `INT_PARAMS` in http.js. Lists that are newest-first (messages)
use the last id as the `after` cursor and return older items; this is the one place `after` does not mean "newer than".
## S3-1 — Online/offline is computed lazily; `powerCheck` only does the side effects
Units are offline while the battery is empty and net power is negative. That is derived from the lazy battery value (`power.online`) at
the moment it matters (sending, combat, the town view), never from an event having run. The `powerCheck` event (one pending per POP,
re-armed by `power.reschedule` after anything that changes unit draw or building levels, including `buildDone`) only writes the
"units offline" notification and does the decommission after 24 h / world speed. Offline is all-or-nothing per POP (the contract of
`rules.unitsOnline`); the decommission then removes the most expensive units (home first, then reinforcements) until the draw fits the
output, not everything. Offline units are neither counted in defence nor hurt by a raid, but a full attack lost by the defender still
wipes them. Reinforcements draw from the host (`state.unitDraw` reads `stationed`); units in transit draw from nobody.
## S3-2 — Research is per POP, speed bonus per operator
`town_research(town_id, unit, stat, level)`; level n needs a Research lab of level n + 2, one research at a time per POP. An attack uses a
snapshot of the origin POP's research taken at departure (`movements.extra.research`); defenders use the research of the POP each group
belongs to. Incumbent sites: +10% research speed per distinct site taken (cap +50%), and a one-time +2500 Hardware per site and operator.
Taking a site does not remove it: it stays a static fortress for everybody else.
## S3-3 — One report row, many recipients; cleanup armed at the earliest expiry
A battle is one `reports` row shared through `report_recipients(report_id, user_id, role, read_at)`, so both sides see the same data (a
scan is two separate rows: intel for the scanner, "a scan happened" for the target, with the scanner's identity only if the target's
scanners won). `reportCleanup` is armed at the earliest `expires_at` (14 days, real time, not divided by world speed) and re-arms itself
only while reports remain: at most one pending cleanup per world, none on a quiet world.
## S3-4 — Arrival re-validates everything and bounces
A movement is validated when it is sent and again when it arrives (the target may have changed owner, got protection, joined a pact, the
tile may have been taken). If it no longer holds, the units turn around with the loot they carry and the sender gets a `movement_failed`
report. Return legs whose origin POP changed hands go to the operator's capital (or oldest POP).
## S3-5 — Takeover details
Only `takeover` movements (with an M&A team) can lower loyalty; one team per attack counts and is used up when it does. `takeoverBlock`:
a player's last POP wins over "capital" (refused at send); a capital with siblings is only dragged to loyalty 0. Loot is taken in the same
arrival, before the POP changes hands. On transfer the old owner's pending training/research is cancelled, pending builds are re-keyed to
the new owner's user, and that POP's units standing elsewhere are re-homed to the old owner's capital. A captured POP keeps its footprint.
## S3-6 — Transport links
`links` = Internet exchange level, each carries 500 (`D.TRANSPORT`). A link is busy while the transport is out and while its empty carriers
drive back (a `return` movement with `extra.link`). Cargo above the receiver's Rack hall cap is lost (the notification says how much).
`transport` movements keep arbitrary `extra` and `movements.hooks.transportArrived` is called on delivery, so the trade stage needs no
change in the movement code.
## S3-7 — Protection
Beginner protection blocks raid/attack/takeover in both directions. Sending one from a protected operator needs `confirmEndProtection`
and ends it at once; attacking an incumbent site does not end it; scans never do. The target's protection is compared with the ARRIVAL
time at send, and re-checked at arrival.
## S3-8 — Footprint gate counts the operator's total
`footprintNeeded(n)` with n = POPs owned now (second POP: n = 1). Founding and takeover both need `sum(footprint of own POPs) >= need`,
nothing is spent. New POPs start with every field at level 0, no infrastructure, 250 of each resource and loyalty 100.
## S3-9 — Moving units between your own POPs
`reinforce` to an own POP puts the units into `town_units` of the target (they live there now). Only units of OTHER operators are
`stationed` rows.
## S6-1 — Push hints leave only after the commit, through an outbox
`scheduler.notify` runs inside the transaction of the event or command, so pushing there would announce changes that may still roll
back. `notify` queues into `scheduler.outbox` (only when a listener is set); `process()` and `commands.run` flush it after COMMIT
(`db.depth === 0`) and truncate it to their mark on rollback. `town` hints use the existing `scheduler.onEvent` (post-commit already)
and a new `commands.hooks.onCommand`. `push.js` itself knows no game rules (it gets `townOwner` / `tokenValid` injected), the routing
lives in `pushHooks.js`. No new dependency (`ws` was already listed).
## S6-2 — Revocation is polled, not pushed
Tokens are revoked by plain SQL in routes and the admin tool. Instead of hooking each place, the ping tick (30 s) re-checks
`tokens.revoked_at` and `users.banned` for every authenticated socket and closes with 4401. Worst case 30 s of hints after logout on
another device; hints carry no data, so nothing leaks. If that is ever not good enough, call a `push.disconnectToken(id)` from the routes.
## S6-3 — Hints are merged per tick, `town` is not sent to the target at launch
Identical JSON hints produced in one tick (a catch-up) are deduplicated per socket via `setImmediate`. A launched movement tells the
target's owner `incoming` (town + ETA) and no plain `town` hint, so the type and origin of a launch cannot be inferred from extra traffic.
## E1 — A landing site is a row of its own, not a POP
`landing_sites(world_id, idx, x, y, holder_alliance_id, holder_operator_id, level, stockpile, build_*)` (migration 004), positions from
`rules.backboneSites(seed, radius)`. A town row of kind `landing` owned by the leader was considered and rejected: a "POP" that nobody plays
would have to be excluded from footprint rankings, the last-POP / capital / footprint-gate counts, `homeTownFor`, `/me`, spawn spacing and
the Rack hall cap (shipped resources would be thrown away at 250), and a leader change would have to re-own it. The holder is the
alliance (`holder_alliance_id`, FK `ON DELETE SET NULL`, the disband hook also clears it), `holder_operator_id` only records who took it.
Consequence, on purpose and to be revisited after the first world: **no units can stand at a site**, so there is no garrison and no
reinforcement. A site is defended by the static value `incumbentDefence x 12 x (1 + 0.03 x level)` only, i.e. exactly the "incumbent
with a backbone multiplier" of DESIGN.md; holders defend by growing the node (higher level = harder to take), attackers can scan the exact
number. Hooks into the military code are small: `combat.arriveSite` hands `backbone` tiles to `military/landing.js`,
`arrivals.arriveTransport` hands shipments (`extra.siteId`) to `endgame.deliverShipment`, `sendMovement` calls `endgame.checkLandingTarget`.
## E2 — Site rows are created lazily, there is no timer event
`ensureSites` creates the 7 rows when something first touches a site (an attack/scan sent or arriving, a shipment, a build) once the world is
at day 100 / speed; that first call also writes a `backbone_open` notification for every operator. Reads (`/worlds/:id/status`, `/map`, tile
info) never write: untouched sites are shown as the incumbent's, level 0, `id: null`. A `backboneOpen` event at world creation was
dropped: a pending event 100 days ahead sits in every quiet world and breaks tooling that counts pending events (the admin CLI test).
Cost: nobody is pushed the moment the sites appear unless a first touch happens; clients see `phase: backbone_open` when they poll status.
## E3 — The winner is fixed at level 100; the world is frozen when it ends
The holder of a node whose level completes at 100 wins (`worlds.winner_*`, once per world) and `worldEnd` is scheduled
`graceDays x 24 h / speed` later. Losing the node meanwhile does not change the winner (the race is over; the grace days are for the ranking
and the post-game). If a level-100 node is taken while nobody holds it (the holder disbanded mid-build) the new holder wins. `worldEnd`
writes the ranking snapshot, sets `status = 'ended'`, closes open trade offers (`expired`, escrow back in its POP: every unit stays accounted
for) and CANCELS every other pending event of the world: nothing arrives, trains,
expires or builds after the snapshot, so the stored ranking stays the truth. Movements still `out` and queued builds stay as they were.
A running build of a site continues for whoever holds the site when it lands (the level belongs to the site; the cost was paid from the
stockpile at the start); the stockpile and the ledger stay with the site when it is taken.
## E4 — `world_ended` is one gate in `commands.run`, found by probing the arguments
`ownedTown`, `operatorOf` and `requireRole` are shared with reads (rankings, map), so gating there would break "readable after the end".
`commands.run` calls `endgame.guard(db, name, args)`: the world is `args.worldId`, else the world of `townId/fromTownId/hostTownId`,
`allianceId`, `movementId`, `inviteId`, `proposalId`, `offerId`, `siteId`, `reportId`, else the caller's oldest operator (the default every
command already uses). Allowed after the end: `markReportRead`, `messageSend/Read/Delete`, `messageBlock/Unblock`. New commands are gated
automatically as long as they carry one of those arguments.
## E5 — Backbone cost curve is not the building curve
`4000 x 1.28^99` is unreachable; the node has its own `ENDGAME.costGrowth = 1.045` and `timeGrowth = 1.025` (about 7.2 M of each resource and
about 36 world days of building in total, level 100 alone 312 k each and 23 h). Pure guesses until a bot run (stage 7) says how much an
alliance can ship; nothing here is balanced.
## T1 — Trade: escrow, one link per side, reach by exchange level
- Offers are escrowed in `trade_offers` and hold one transport link of the poster's POP while open (`movements.activeLinks` counts open
offers), so accepting never fails for lack of a link on the poster's side and cannot exceed the exchange level.
- Each side of an offer must fit ONE transport (<= 500); larger deals are several offers. The ratio band is count-based 1:2.
- Escrow returns without the Rack hall cap (conservation beats the cap here); deliveries keep the existing cap rule (`cargoLost`).
- Reach: `10 tiles x exchange level` of the taker's POP, and the offer's own `maxDistance`; the poster's level only matters for the number of
offers they can hold open.
- A POP changing hands voids its open offers (`combat.transferTown` calls `trade.onTownTransferred`); the escrow stays in the POP.
- `transportArrived` hook: `server/trade.js` registers itself when required (both `commands/index.js` and `events/index.js` load it), so the
consumer exists in every test fixture that builds commands or events.
- Offer expiry: one `tradeExpire` event per offer, cancelled on accept/cancel/void, idempotent (acts only while `open`).
## T2 — Shared reports extend the report's life
Sharing sets `expires_at = max(expires_at, now + 14 days)` on the single shared `reports` row, so the folder is not emptied by a report that
was nearly due; the cascade `report_shares.report_id ... ON DELETE CASCADE` removes the share when the report is cleaned up.
## G1 — `buildDone` adds one level to what the building is NOW
Each build event adds exactly one level to the current level (never lowers, never above the queued `toLevel`), so an upgrade in flight
cannot undo exploit-kit damage; a building knocked to 0 comes back at 1. The notification reports the real level.
## G2 — Allied incoming is a separate list
`GET /movements` keeps `incoming` as "heading for YOUR POPs" and adds `alliedIncoming` for partners' POPs (own alliance + confederations),
each entry with `ally: true` and `toOperator`. Mixing them would make clients count allies' attacks as their own.
## R1 — Review round 1 (time model + security): what was fixed and what was accepted
Fixed (each with a regression test in `test/review-fixes.test.js`): `__proto__`/`constructor` keys smuggling cargo past the transport
capacity check (validator now rejects them at any depth); trade legs recallable so one side kept both halves of a deal (trade/site legs
are no longer recallable); trade escrow refunds lifting stock above the Rack hall cap; static files served without CSP/nosniff/
no-referrer and a `connect-src` that allowed any `ws:`/`wss:` host (now `'self'`); IPv6 clients minting unlimited rate-limit buckets
(one bucket per /64), the limiter forgetting throttled keys when full (LRU eviction), no per-username login throttle (added), scrypt CPU DoS
(max 4 concurrent hashes, username checked before hashing); rankings/alliance reads scanning the whole world per request (5 s footprint
cache per world); weaker town-name sanitiser (now `text.cleanText`); rejected 429s pinned into the idempotency store (no longer);
notifications never pruned (30 days); `units_offline` warning missing when units went offline through an arrival.
Accepted for now, written down so nobody is surprised:
- Escrowed trade stock is not lootable, so an unattractive offer is a hiding place for up to 250-500 of one resource per link (48 h
offers, cancellable). Revisit if bots or players abuse it.
- Leaving/being kicked from an alliance drops pact protection at once (pacts bind alliances, not people). Revisit: track `left_at`.
- A battery above a lowered Battery bank cap is clamped immediately, while stock above a lowered Rack hall cap is kept. Intentional:
power is a live balance, stock is a physical pile.
- If an `arrive`/`return` handler ever throws, the event is marked failed and the movement stays `out` (units lost). No trigger found by
the reviewer or ~70 fuzz seeds; a recovery sweeper is a stage-7 item.
- HTTP connections per IP are not capped (only WebSockets are); behind nginx Proxy Manager the per-IP request limiter applies.
- Tokens do not expire and there is no change-password route yet (revoke from Settings works).