Phase 4: close loot/refund leaks, landing-site garrisons, speed-scaled shipping, update/offsite tooling, broad e2e

Built:
- Economy: raid loot capped to the room the raiders have at home (rest stays with the victim, "lootLeft" in the report); trade escrow reserves its room so refunds always fit; trade accept refuses with no_room_for_goods when the goods would not fit. Loot destroyed per 30-day run 31-61k -> ~1k, refunds clipped ~27k -> 0 (docs/BALANCE.md, 3 seeds).
- End game: members of the holding alliance garrison landing sites (migration 005_site_garrison.sql), garrison counts in site defence, 72 h capture shield, withdraw route, POP view lists garrisoned units. Shipping capacity scales with world speed (3x world: winner 76-98 -> ~30-36 days after sites open). Bot runs show real contests (84-194 attacks on held sites at 1x).
- Ops: deploy/update.sh (backup, stop, ff-merge, npm ci --ignore-scripts, health check, rollback), deploy/upstream-offsite (rsync only), admin end-world snapshots results, EPIPE fix, docs/workflows/ filled in.
- e2e 62 -> 163 checks: units, raids/reports, alliance+messages, trade, end game, nb, PWA, reconnect, slow polling fallback (opt-in). Two app bugs found: message push ignored, "New" button stayed English.

Tested: npm test 274/274, openapi check, npm run e2e 163/163 (137 s), bot runs 50x30d seeds 1-3 and 140-day/60-day speed-3 end-game runs INVARIANTS OK, adversarial review pass (findings fixed or documented).
NOT tested: update.sh on a real box (systemd, real npm, runuser; stubs only, shellcheck not installed), garrison card/loot text on a phone, Node 22.13, humans. Known leaks left open and documented (BALANCE S6): loot/shipments can still overflow when stock arrives in flight, bounced backbone shipments lose overflow (reported to the sender). Garrison numbers (heldDefenceMult 2, shield 72 h) are first guesses from 2 seeds. Not claimed: balanced.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q2MJaQ5LXAHzGnTDgceT14
This commit is contained in:
jvanvik 2026-10-12 00:58:12 +02:00
parent 9104480463
commit 3c23c51902
92 changed files with 4887 additions and 300 deletions

69
deploy/upstream-offsite Executable file
View file

@ -0,0 +1,69 @@
#!/bin/sh
# Copy the newest VERIFIED database snapshot off the box. Started by upstream-offsite.service (timer), or by hand with
# `systemctl start upstream-offsite` (that gives it the user, the env file and the sandbox). Nothing happens unless
# OFFSITE_DEST is set (in /etc/upstream/upstream.env); docs/DEPLOY.md section 7 "Off-box copy".
#
# OFFSITE_DEST required. rsync destination DIRECTORY: user@host:/srv/backups/upstream/ or a local path (a mounted NAS)
# OFFSITE_METHOD rsync (the only method). scp was removed: it cannot refuse to overwrite a same-named file on the far side
# OFFSITE_SSH_KEY optional private key file (owned by the service user, mode 0600); used with IdentitiesOnly
# OFFSITE_KNOWN_HOSTS known_hosts file for the destination (default /etc/upstream/offsite_known_hosts when it exists);
# host keys are always checked (StrictHostKeyChecking=yes), BatchMode=yes: never a prompt
# OFFSITE_SSH_OPTS extra ssh options, e.g. "-p 2222"
# BACKUP_DIR where snapshots are written (default /var/lib/upstream/backups)
# UPSTREAM_ADMIN admin wrapper used for verify-backup (default upstream-admin)
#
# upstream-offsite --check exit 0 when OFFSITE_DEST is set, 1 when not (the unit's ExecCondition: unset = skipped quietly)
#
# Steps: pick the newest snapshot (by modification time) -> verify-backup it (integrity_check, schema present) -> copy it, never
# overwriting a file of the same name on the far side -> compare content with --checksum. Old snapshots on the far side
# are never deleted by this script (prune there, or set up your own retention).
# Exit codes: 0 copied (or already there), 1 failed / nothing to copy / not configured under --check, 2 usage or configuration error.
set -eu
BACKUP_DIR=${BACKUP_DIR:-/var/lib/upstream/backups}
ADMIN=${UPSTREAM_ADMIN:-upstream-admin}
METHOD=${OFFSITE_METHOD:-rsync}
DEST=${OFFSITE_DEST:-}
KEY=${OFFSITE_SSH_KEY:-}
KNOWN=${OFFSITE_KNOWN_HOSTS:-/etc/upstream/offsite_known_hosts}
SSH_EXTRA=${OFFSITE_SSH_OPTS:-}
say() { printf '%s\n' "offsite: $*"; }
die() { printf '%s\n' "offsite: $*" >&2; exit "${2:-1}"; }
case ${1:-} in
--check) [ -n "$DEST" ]; exit $? ;;
'') ;;
*) die "usage: upstream-offsite [--check]" 2 ;;
esac
[ -n "$DEST" ] || die "OFFSITE_DEST is not set: off-box copy is not configured (docs/DEPLOY.md section 7)" 2
case $METHOD in
rsync) ;;
scp) die "OFFSITE_METHOD=scp is not supported any more: scp cannot refuse to overwrite a file of the same name on the far side. Use rsync (the far side needs rsync, or a forced rrsync command)" 2 ;;
*) die "OFFSITE_METHOD must be rsync, not '$METHOD'" 2 ;;
esac
[ -d "$BACKUP_DIR" ] || die "no backup directory $BACKUP_DIR"
# newest snapshot by mtime (names sort by time too, but a same-second '-1' file would sort wrongly)
file=
for f in $(ls -1t "$BACKUP_DIR" 2>/dev/null); do
case $f in upstream-[0-9]*T[0-9]*Z.db|upstream-[0-9]*T[0-9]*Z-[0-9]*.db) file=$BACKUP_DIR/$f; break ;; esac
done
[ -n "$file" ] || die "no snapshot (upstream-*.db) in $BACKUP_DIR: nothing to copy"
"$ADMIN" verify-backup "$file" >/dev/null || die "newest snapshot $file does not verify: NOT copied (the daily backup unit should have failed too: journalctl -u upstream-backup)"
SSH_OPTS="-o BatchMode=yes -o StrictHostKeyChecking=yes"
[ -z "$KEY" ] || SSH_OPTS="$SSH_OPTS -i $KEY -o IdentitiesOnly=yes"
[ ! -f "$KNOWN" ] || SSH_OPTS="$SSH_OPTS -o UserKnownHostsFile=$KNOWN"
[ -z "$SSH_EXTRA" ] || SSH_OPTS="$SSH_OPTS $SSH_EXTRA"
SSH="ssh $SSH_OPTS" # (paths in these settings must not contain spaces)
dest=${DEST%/}/
name=$(basename "$file")
# --ignore-existing: a snapshot name is a moment in time, a same-named file on the far side is never replaced
rsync -t --chmod=F600 --ignore-existing -e "$SSH" "$file" "$dest" || die "rsync to $DEST failed"
# content check of what is over there now (empty output = identical bytes)
diff=$(rsync -c -n -i -e "$SSH" "$file" "$dest" 2>&1) || die "could not re-read $DEST to compare: $diff"
[ -z "$diff" ] || die "far-side copy of $name differs from the verified local snapshot ($diff): a different file of that name already exists there"
say "ok: $name is on $DEST (checksum compared)"