Upstream/docs/DECISIONS.md
jvanvik cc0524de5e Smoke e2e, docs consistency pass, status summary
Built: e2e/run.js + lib.js (playwright-core, in-process server, temp DB): desktop flow, phone touch portrait/landscape, layout matrix on 5 viewports x pop/map/trade/world (overflow + console errors), WebSocket push incoming-banner without reload. 61/61 checks, ~54 s, identical over 4 runs; the push and overflow checks were shown to fail when the behaviour is removed.
Docs: fixed stale numbers/claims in DESIGN, DECISIONS, BALANCE, API (undocumented routes, rate limits, notification types), DEPLOY, NEXT (status-at-a-glance on top). Deploy path exercised in a scratch dir: migrations, healthz, signup/build, admin CLI, SIGKILL catch-up, backup restore.

Tested: npm test 207/207, npm run e2e 61/61, bot run 50x14d INVARIANTS OK.
NOT tested: Chromium only; no systemd/nginx Proxy Manager/Node 22.13 run; e2e does not cover units, alliances, messages, trade or end-game flows beyond empty-state rendering; no nb, PWA or polling-fallback e2e; docs/workflows/ still empty.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2MJaQ5LXAHzGnTDgceT14
2026-10-11 20:34:35 +02:00

23 KiB

DECISIONS

Short log of non-obvious decisions and why. Newest at the bottom.

D1 — node:sqlite as the driver

Dev machine runs Node 24; the deploy target (Debian 13 LXC) ships Node 22+. node:sqlite is used through server/db.js only, so better-sqlite3 is a one-file swap if it ever misbehaves. Parameters are always positional.

D2 — Scheduler processes ALL due events before any command or read

scheduler.runDue() runs every pending event with due_at <= now globally, in (due_at, id) order, not just the ones touching the town in question. It is simpler and makes fairness trivial (nothing resolves out of order). Cost is nil in practice because the timer processes events within milliseconds of their due time; it only matters after a restart (catch-up).

D3 — One towns.res_at for all lazily-grown values

Fiber, hardware, capital, power battery, footprint and loyalty are all settled together at res_at (state.settle). Any change that alters a rate (building finished, unit trained or lost) settles first. rules.resourcesAt is therefore exact between settles.

D4 — Power battery uses the same lazy model

Battery = (amount, rate = net power x world speed, cap = Battery bank). Units are offline while battery = 0 and net < 0 (rules.unitsOnline). The 24 h decommission countdown is an event scheduled from rules.batteryEmptyAt.

D5 — Rural areas are held by proximity in v1

DESIGN.md leaves "contested or just held" open. v1: a POP within ruralReach (2) tiles of a rural tile gets its +25% (max 3, nearest first). Computed from the seed, no state. Revisit after the first world.

D6 — First world is 101 x 101, not 301 x 301

DESIGN.md's own open question. mapRadius is world data (worlds.map_radius), default 50.

D7 — docs/openapi.json is generated

tools/gen-openapi.js builds it from the route table (schemas live in route meta + server/routes/schemas.js). npm test fails when the committed file is stale, which satisfies "updated in the same commit as any route change" mechanically.

D8 — Resource fields are slot-keyed

Buildings table key: f1..f18 for resource fields (type fixed at founding, 4 per type + 2 wild), the building type for infrastructure. Build "slots": 1 concurrent build, 2 from Core router 3 (was 10, changed by B1; queue = pending buildDone events).

S5-1 — Alliance size cap is stored, only ever raised

DESIGN says the Peering office level sets the size you can found. The cap is stored at founding (10 + 2 * best level). A lost or downgraded building must never kick anyone, so the cap is monotonic: commands that depend on it (invite, accept, transfer) raise it if the CURRENT leader's best office would give more; reads show max(stored, formula) without writing. The floor is therefore 10 even for a leader without an office after a transfer. The office requirement (>= 1) is checked when founding and when accepting an invite, not afterwards.

S5-2 — Diplomacy move table is data, three kinds of change

SOCIAL.diplomacyMoves (data.js) says for each (from, to) whether it is immediate, a proposal or delayed. War from neutral is immediate (no consent needed). Anything that leaves a pact (nap/confed -> neutral/war, confed -> nap) is delayed: 24 h / world speed via a diploChange event, so you cannot betray a partner without warning. Pacts and peace need acceptance. While a delayed change is pending the pair is locked (diplomacy_pending); only the starter can revoke. A pair at neutral with nothing pending has no row in alliance_relations. Acceptance re-validates against the current state, because the relation may have moved since the proposal. war -> confed is not allowed (go through a pact first).

S5-3 — Membership lives in two tables, one writer

operators.alliance_id (from 001, cheap joins in /map, /me) and alliance_members (role, joined_at, one leader per alliance by a partial unique index). alliances.js addMember/removeMember are the only writers. Alliance rows are hard-deleted on disband; messages keep a alliance_tag snapshot and carry no foreign key to alliances so broadcasts survive.

S5-4 — Last online comes from tokens

No new column: lastOnline = newest tokens.last_used_at of the member's user, which auth.js refreshes at most once a minute. So it is accurate to about a minute and includes revoked tokens (a logout still counts as having been online).

S5-5 — Message rate limit is DB-backed, not the in-memory Limiter

server/http.js Limiter keys on monotonic real time and forgets on restart; message limits are a product rule that should follow the (fake) clock and survive restarts, so the sender's recent rows in messages are counted (SOCIAL.messageLimits, indexed by messages_sender). The per-token/IP limiters still apply on top.

S5-6 — Messages: per-recipient rows, sanitising on write

One messages row plus one message_recipients row per recipient (broadcast fan-out at send time: members who join later do not see old broadcasts; members who leave keep what they received). Read/delete state is per recipient; the sender's copy is hidden with sender_deleted_at; the row is purged when nobody has a copy. A blocked sender is silently not delivered to (no row), so blocking does not leak. Text is sanitised once on write (server/text.js): C0/C1 controls, bidi overrides and isolates, zero-width and invisible tag characters are stripped, NFC, \r\n -> \n; ZWJ/ZWNJ are kept for emoji. Clients render with textContent only.

S5-7 — Rankings are computed per request, no cache

Footprint for a whole world comes from two queries (towns, buildings) and rules.footprintAt; fine for the expected 50-300 operators. If it ever shows up in a profile, cache per (world, minute). Cursor is a rank offset because rankings are a computed snapshot, not rows.

S5-8 — Route params and list order

inviteId, noteId, proposalId, otherId, operatorId were added to INT_PARAMS in http.js. Lists that are newest-first (messages) use the last id as the after cursor and return older items; this is the one place after does not mean "newer than".

S3-1 — Online/offline is computed lazily; powerCheck only does the side effects

Units are offline while the battery is empty and net power is negative. That is derived from the lazy battery value (power.online) at the moment it matters (sending, combat, the town view), never from an event having run. The powerCheck event (one pending per POP, re-armed by power.reschedule after anything that changes unit draw or building levels, including buildDone) only writes the "units offline" notification and does the decommission after 24 h / world speed. Offline is all-or-nothing per POP (the contract of rules.unitsOnline); the decommission then removes the most expensive units (home first, then reinforcements) until the draw fits the output, not everything. Offline units are neither counted in defence nor hurt by a raid, but a full attack lost by the defender still wipes them. Reinforcements draw from the host (state.unitDraw reads stationed); units in transit draw from nobody.

S3-2 — Research is per POP, speed bonus per operator

town_research(town_id, unit, stat, level); level n needs a Research lab of level n + 2, one research at a time per POP. An attack uses a snapshot of the origin POP's research taken at departure (movements.extra.research); defenders use the research of the POP each group belongs to. Incumbent sites: +10% research speed per distinct site taken (cap +50%), and a one-time +2500 Hardware per site and operator. Taking a site does not remove it: it stays a static fortress for everybody else.

S3-3 — One report row, many recipients; cleanup armed at the earliest expiry

A battle is one reports row shared through report_recipients(report_id, user_id, role, read_at), so both sides see the same data (a scan is two separate rows: intel for the scanner, "a scan happened" for the target, with the scanner's identity only if the target's scanners won). reportCleanup is armed at the earliest expires_at (14 days, real time, not divided by world speed) and re-arms itself only while reports remain: at most one pending cleanup per world, none on a quiet world.

S3-4 — Arrival re-validates everything and bounces

A movement is validated when it is sent and again when it arrives (the target may have changed owner, got protection, joined a pact, the tile may have been taken). If it no longer holds, the units turn around with the loot they carry and the sender gets a movement_failed report. Return legs whose origin POP changed hands go to the operator's capital (or oldest POP).

S3-5 — Takeover details

Only takeover movements (with an M&A team) can lower loyalty; one team per attack counts and is used up when it does. takeoverBlock: a player's last POP wins over "capital" (refused at send); a capital with siblings is only dragged to loyalty 0. Loot is taken in the same arrival, before the POP changes hands. On transfer the old owner's pending training/research is cancelled, pending builds are re-keyed to the new owner's user, and that POP's units standing elsewhere are re-homed to the old owner's capital. A captured POP keeps its footprint.

links = Internet exchange level, each carries 1000 (D.TRANSPORT; 500 until B2). A link is busy while the transport is out and while its empty carriers drive back (a return movement with extra.link). Cargo above the receiver's Rack hall cap is lost (the notification says how much). transport movements keep arbitrary extra and movements.hooks.transportArrived is called on delivery, so the trade stage needs no change in the movement code.

S3-7 — Protection

Beginner protection blocks raid/attack/takeover in both directions. Sending one from a protected operator needs confirmEndProtection and ends it at once; attacking an incumbent site does not end it; scans never do. The target's protection is compared with the ARRIVAL time at send, and re-checked at arrival.

S3-8 — Footprint gate counts the operator's total

footprintNeeded(n) with n = POPs owned now (second POP: n = 1). Founding and takeover both need sum(footprint of own POPs) >= need, nothing is spent. New POPs start with every field at level 0, no infrastructure, 250 of each resource and loyalty 100.

S3-9 — Moving units between your own POPs

reinforce to an own POP puts the units into town_units of the target (they live there now). Only units of OTHER operators are stationed rows.

S6-1 — Push hints leave only after the commit, through an outbox

scheduler.notify runs inside the transaction of the event or command, so pushing there would announce changes that may still roll back. notify queues into scheduler.outbox (only when a listener is set); process() and commands.run flush it after COMMIT (db.depth === 0) and truncate it to their mark on rollback. town hints use the existing scheduler.onEvent (post-commit already) and a new commands.hooks.onCommand. push.js itself knows no game rules (it gets townOwner / tokenValid injected), the routing lives in pushHooks.js. No new dependency (ws was already listed).

S6-2 — Revocation is polled, not pushed

Tokens are revoked by plain SQL in routes and the admin tool. Instead of hooking each place, the ping tick (30 s) re-checks tokens.revoked_at and users.banned for every authenticated socket and closes with 4401. Worst case 30 s of hints after logout on another device; hints carry no data, so nothing leaks. If that is ever not good enough, call a push.disconnectToken(id) from the routes.

S6-3 — Hints are merged per tick, town is not sent to the target at launch

Identical JSON hints produced in one tick (a catch-up) are deduplicated per socket via setImmediate. A launched movement tells the target's owner incoming (town + ETA) and no plain town hint, so the type and origin of a launch cannot be inferred from extra traffic.

E1 — A landing site is a row of its own, not a POP

landing_sites(world_id, idx, x, y, holder_alliance_id, holder_operator_id, level, stockpile, build_*) (migration 004), positions from rules.backboneSites(seed, radius). A town row of kind landing owned by the leader was considered and rejected: a "POP" that nobody plays would have to be excluded from footprint rankings, the last-POP / capital / footprint-gate counts, homeTownFor, /me, spawn spacing and the Rack hall cap (shipped resources would be thrown away at 250), and a leader change would have to re-own it. The holder is the alliance (holder_alliance_id, FK ON DELETE SET NULL, the disband hook also clears it), holder_operator_id only records who took it. Consequence, on purpose and to be revisited after the first world: no units can stand at a site, so there is no garrison and no reinforcement. A site is defended by the static value incumbentDefence x 12 x (1 + 0.03 x level) only, i.e. exactly the "incumbent with a backbone multiplier" of DESIGN.md; holders defend by growing the node (higher level = harder to take), attackers can scan the exact number. Hooks into the military code are small: combat.arriveSite hands backbone tiles to military/landing.js, arrivals.arriveTransport hands shipments (extra.siteId) to endgame.deliverShipment, sendMovement calls endgame.checkLandingTarget.

E2 — Site rows are created lazily, there is no timer event

ensureSites creates the 7 rows when something first touches a site (an attack/scan sent or arriving, a shipment, a build) once the world is at day 100 / speed; that first call also writes a backbone_open notification for every operator. Reads (/worlds/:id/status, /map, tile info) never write: untouched sites are shown as the incumbent's, level 0, id: null. A backboneOpen event at world creation was dropped: a pending event 100 days ahead sits in every quiet world and breaks tooling that counts pending events (the admin CLI test). Cost: nobody is pushed the moment the sites appear unless a first touch happens; clients see phase: backbone_open when they poll status.

E3 — The winner is fixed at level 100; the world is frozen when it ends

The holder of a node whose level completes at 100 wins (worlds.winner_*, once per world) and worldEnd is scheduled graceDays x 24 h / speed later. Losing the node meanwhile does not change the winner (the race is over; the grace days are for the ranking and the post-game). If a level-100 node is taken while nobody holds it (the holder disbanded mid-build) the new holder wins. worldEnd writes the ranking snapshot, sets status = 'ended', closes open trade offers (expired, escrow back in its POP: every unit stays accounted for) and CANCELS every other pending event of the world: nothing arrives, trains, expires or builds after the snapshot, so the stored ranking stays the truth. Movements still out and queued builds stay as they were. A running build of a site continues for whoever holds the site when it lands (the level belongs to the site; the cost was paid from the stockpile at the start); the stockpile and the ledger stay with the site when it is taken.

E4 — world_ended is one gate in commands.run, found by probing the arguments

ownedTown, operatorOf and requireRole are shared with reads (rankings, map), so gating there would break "readable after the end". commands.run calls endgame.guard(db, name, args): the world is args.worldId, else the world of townId/fromTownId/hostTownId, allianceId, movementId, inviteId, proposalId, offerId, siteId, reportId, else the caller's oldest operator (the default every command already uses). Allowed after the end: markReportRead, messageSend/Read/Delete, messageBlock/Unblock. New commands are gated automatically as long as they carry one of those arguments.

E5 — Backbone cost curve is not the building curve

4000 x 1.28^99 is unreachable; the node has its own ENDGAME.costGrowth = 1.045 and timeGrowth = 1.025 (about 7.2 M of each resource and about 36 world days of building in total, level 100 alone 312 k each and 23 h). Pure guesses until a bot run (stage 7) says how much an alliance can ship; nothing here is balanced.

  • Offers are escrowed in trade_offers and hold one transport link of the poster's POP while open (movements.activeLinks counts open offers), so accepting never fails for lack of a link on the poster's side and cannot exceed the exchange level.
  • Each side of an offer must fit ONE transport (<= 1000, was 500 until B2); larger deals are several offers. The ratio band is count-based 1:2.
  • Escrow returned WITHOUT the Rack hall cap at first (conservation beats the cap); review round 1 (R1) reversed that: refunds are capped like cancelBuild (stock already above the cap stays, the clipped part is lost silently, BALANCE.md S2). Deliveries keep the cap rule (cargoLost).
  • Reach: 10 tiles x exchange level of the taker's POP, and the offer's own maxDistance; the poster's level only matters for the number of offers they can hold open.
  • A POP changing hands voids its open offers (combat.transferTown calls trade.onTownTransferred); the escrow stays in the POP.
  • transportArrived hook: server/trade.js registers itself when required (both commands/index.js and events/index.js load it), so the consumer exists in every test fixture that builds commands or events.
  • Offer expiry: one tradeExpire event per offer, cancelled on accept/cancel/void, idempotent (acts only while open).

T2 — Shared reports extend the report's life

Sharing sets expires_at = max(expires_at, now + 14 days) on the single shared reports row, so the folder is not emptied by a report that was nearly due; the cascade report_shares.report_id ... ON DELETE CASCADE removes the share when the report is cleaned up.

G1 — buildDone adds one level to what the building is NOW

Each build event adds exactly one level to the current level (never lowers, never above the queued toLevel), so an upgrade in flight cannot undo exploit-kit damage; a building knocked to 0 comes back at 1. The notification reports the real level.

G2 — Allied incoming is a separate list

GET /movements keeps incoming as "heading for YOUR POPs" and adds alliedIncoming for partners' POPs (own alliance + confederations), each entry with ally: true and toOperator. Mixing them would make clients count allies' attacks as their own.

R1 — Review round 1 (time model + security): what was fixed and what was accepted

Fixed (each with a regression test in test/review-fixes.test.js): __proto__/constructor keys smuggling cargo past the transport capacity check (validator now rejects them at any depth); trade legs recallable so one side kept both halves of a deal (trade/site legs are no longer recallable); trade escrow refunds lifting stock above the Rack hall cap; static files served without CSP/nosniff/ no-referrer and a connect-src that allowed any ws:/wss: host (now 'self'); IPv6 clients minting unlimited rate-limit buckets (one bucket per /64), the limiter forgetting throttled keys when full (LRU eviction), no per-username login throttle (added), scrypt CPU DoS (max 4 concurrent hashes, username checked before hashing); rankings/alliance reads scanning the whole world per request (5 s footprint cache per world); weaker town-name sanitiser (now text.cleanText); rejected 429s pinned into the idempotency store (no longer); notifications never pruned (30 days); units_offline warning missing when units went offline through an arrival. Accepted for now, written down so nobody is surprised:

  • Escrowed trade stock is not lootable, so an unattractive offer is a hiding place for up to 1000 (500 before B2) of one resource per link (48 h offers, cancellable). Revisit if bots or players abuse it.
  • Leaving/being kicked from an alliance drops pact protection at once (pacts bind alliances, not people). Revisit: track left_at.
  • A battery above a lowered Battery bank cap is clamped immediately, while stock above a lowered Rack hall cap is kept. Intentional: power is a live balance, stock is a physical pile.
  • If an arrive/return handler ever throws, the event is marked failed and the movement stays out (units lost). No trigger found by the reviewer or ~70 fuzz seeds; a recovery sweeper is a stage-7 item.
  • HTTP connections per IP are not capped (only WebSockets are); behind nginx Proxy Manager the per-IP request limiter applies.
  • Tokens do not expire and there is no change-password route yet (revoke from Settings works).

B1 — Second build slot at Core router 3, and slots are data (stage 7 balance pass, docs/BALANCE.md)

The bot harness (tools/bot.js) showed that progress is limited by how many builds a player can START per sitting, not by resources or build time (builds take minutes, a level costs hours of production; with one slot a 5-minute sitting starts one or two). 2 sittings a day needed 62 days for the capital's mean field level to reach 10, 4 a day 32. FORMULAS.secondQueueCoreLevel 10 -> 3 (data only) brings that to 39 and 22 days and changes nothing for a player who is always online. rules.queueSlots now also reads FORMULAS.extraQueueCores (default []: more slots at the listed Core router levels; behaviour unchanged until someone fills it). That is a three-line extension of a formula, not a bug fix, logged here because the rule was "formulas only if wrong": more slots (2/3/4/5 at Core 2/4/7/10) would remove most of the gap between casual and committed players but saturate the committed ones six weeks before the backbone opens; left as a pacing decision.

The end game is shipping-limited, not production-limited: 21.6 M units (7.2 M of each resource) in 500-unit trips. In three seeds with 50 bots (4 to 5 sittings a day, alliances of 13 to 19) nobody finished level 100 by day 175 to 190 at speed 1 (levels 88 to 97); with 1000 per link one alliance won in each seed, 55 to 78 days after the sites opened (node build time alone is 36 days). Raising ENDGAME.costGrowth to 1.035 or the capacity to 2000 were measured and not taken (2000 leaves only the 36-day build floor as a constraint). Not solved: a 3x world takes about as long after the sites open as a 1x world, because shipping is per sitting, not per hour; capacityPerLink x world speed would fix it but the client reads the constant. See BALANCE.md section 5.

B3 — A bounced shipment reports its overflow

events/return.js stayed silent for empty-carrier return legs AND for shipments that came home with cargo; cargo above the Rack hall cap is dropped on arrival, so that loss was invisible. Link legs that carry cargo now send movement_returned with cargoLost (test/bot-server-fixes.test.js). Found by the harness's conservation ledger (BALANCE.md section 6, S1).