Server-run play, part 2: the engine

The engine that runs every network of a server session (server-sim design,
section 4): each network in its own vm realm, realms in a small pool of
worker threads, a promise API for the session host. Nothing uses it yet;
the plain build and the game are unchanged (no file under src/).

- scripts/lib/mp-sim-realm.js: one network in a DONT_CONTEXTIFY realm
  running the plain page's game (never the MP page's client folder), with
  recording DOM stand-ins, no-op timers that count, and the srv... calls:
  load from a checkpoint (route memory seeded, then routed), begin a new
  network, marks and restores, the day as a page runs it, the action
  pipeline with drafts, arming, the sanitize gate and rollback, what-ifs
  that put S, R, the dice and uidN back, pulls with top-level key diffs,
  slim R and flow records on demand, fx captured and bounded, saves, cards.
- scripts/lib/mp-sim-worker.js: realms per session in one worker, days one
  network at a time with other messages between them, loads and unloads
  waiting for a session's day, time limits; a throw goes back to the mark
  in place, a timeout gets a fresh realm (new epoch) carrying the old R, a
  day is retried once and the network broken after a second failure.
- scripts/lib/mp-sim-engine.js: createEngine() with the fake engine's API,
  placement by realm count, every pending call rejected when its worker
  exits, a worker of its own after two exits in an hour; snapWith() joins
  the host's clock, roster and since to a realm's snapshot.
- Contract: a pulled snapshot is the realm's half (the host adds clock,
  roster, since); haptic fx carry milliseconds; the fake engine follows,
  with close() and snapWith(). The button crawl moved to tests/lib.
- Tests: realm vs jsdom byte for byte over 60 scenic big-network days with
  actions; restore; retry after a throw or a timeout; realm order; a timer
  audit and the gate over a sweep of sent buttons (SIM_LONG=1: a year and
  every button); captures, rollbacks, asks, selection reports, ordering,
  bounds, pollution, worker exits; the act tables against the game.

The gate found two keys the game writes in a shape a reload repairs (buyTE's
S.te, mkLeave's S.monkey): game fixes for the integrator, kept visible as a
todo test and a list in the audit.

Rebased onto the 3.3.0 merge (61e31c5): the engine itself needed no change. A
realm still routes in the game's own code (no route worker, no saves), the
route cache that now keeps list orders from day to day leaves a checkpoint and a
retry the same, and the capacity market and moved customers play in a realm as
on a page, under the strict gate. The tests needed three things: an old save's
first days earn the new traffic badges, which the unwatched network's snapshot
keeps; the restore test compares JSON text with sorted keys instead of a
deep-equal of two networks (a diff of gigabytes when they differ); and the crawl
opens the windows that move a customer, with a new test playing the market's and
the path moves' acts, 40 days of sales and a checkpoint against the page.
This commit is contained in:
jvanvik 2026-11-07 11:21:23 +01:00
parent 61e31c53d3
commit 065c922287
12 changed files with 1906 additions and 29 deletions

View file

@ -0,0 +1,182 @@
// The server-run engine (the server-sim design, sections 4.2-4.3): a small pool of worker threads (mp-sim-worker.js),
// each holding the vm realms (mp-sim-realm.js) of the sessions placed on it, behind a promise API. The session host
// (scripts/lib/mp-sim.js) keeps the clock, presence and votes, and asks this for days, actions, what-ifs and pulls.
// tests/lib/mp-sim-fake-engine.js has the same API with canned answers.
//
// const engine = createEngine({workers: 2, heapMb: 1536, root, log});
// engine.load(code, [{nid, env?, begin?, lang, prefs, rng?}]) → [{t:'loaded', nid, epoch, day, lost, saved?} |
// {t:'loaded', nid, err}], one per network, in order. env: the checkpoint envelope {s, uidN, rt, last,
// day} (s: the save text); begin: {name, home, preset, asn, mp, tut} builds a new network, checkpointed
// at once (saved). A network whose save does not load (or that cannot begin) answers err: broken (4.7).
// engine.day(code, {d, speed, last, alone:[nid], nets:[nid], mute}) → {t:'dayDone', code, d, nets:{[nid]: {day, lost,
// pause?, err?, retried?, ms}}}. Each network in sorted order; one that failed twice answers err and
// its day as it was, and is broken from then on (no more days; actions answer 'stopped').
// engine.act(code, nid, msg) → {t:'actDone', nid, epoch, rid, outcome, why?, v, upd} (upd: the update with the result,
// version v; null when stopped) | {t:'err', code, nid, rid, what:'timeout'|'throw', msg}
// engine.ask(code, nid, msg) → {t:'askDone', nid, rid, modal?, ui?, err?} | {t:'err', …}
// engine.pull(code, nid, {snap, views}) → {t:'pulled', nid, epoch, v, upd?, snap?, subs?}. snap is the realm's half
// of a snapshot: snapWith(snap, {clock, roster, since}) makes the stream event.
// engine.sub(code, nid, {views}) → {t:'subbed', nid, epoch, v, subs}; engine.greet(code, nid) → a pulled
// engine.prefs(code, nid, {prefs}) → undefined; engine.save(code, nid) → {t:'saved', nid, txt, uidN, rt, day, lost}
// engine.card(code, nid) → {t:'card', nid, card}; engine.unload(code, nid?) → {t:'unloaded', code, nid?}
// engine.stats() → {workers, realms, sessions} (not a promise); engine.close() → stops the workers
// engine.simId the build id this engine's game belongs to (SIM_ID of the multiplayer page from the same src/)
//
// Rejections: a call for a network that is not loaded (code 'NO_REALM'), a malformed call, and every call pending on
// a worker that exits (out of memory, a crash): Error('worker exited'), code 'EXIT'; that worker's sessions are gone and
// the host loads them again (4.7). A session whose worker exited twice within an hour gets a worker of its own; when that
// one exits too, its loads reject with code 'GAVE_UP' until the engine is made again (the server restarts).
// Epochs: every realm load mints one (a load, a begin, the fresh realm after a timeout); loaded, pulled, actDone and
// subbed carry the realm's current one. Versions are the realm's: only a pull (or an action's own pull) makes one.
//
// Tests only, with createEngine({test: {seed, now, step} | true}): engine.eval(code, nid, src) runs code in a realm
// ({value} | {error} | {timeout, epoch}); engine.mem() reports each worker's heap; engine.kill(i) ends worker i.
'use strict';
const path = require('node:path');
const vm = require('node:vm');
const { Worker } = require('node:worker_threads');
const FILE = path.join(__dirname, 'mp-sim-worker.js');
const HOUR = 3600e3;
/* The stream event of a snapshot: the realm's half (S, r, keep, epoch, v, now) with the host's (clock, roster,
since), joined as text: the realm's 270 KB are never parsed again. */
function snapWith(snap, { clock, roster, since }) {
if (typeof snap !== 'string' || !snap.endsWith('}')) throw new Error('snapWith: not a snapshot');
return snap.slice(0, -1) + ',"clock":' + JSON.stringify(clock) + ',"roster":' + JSON.stringify(roster) + ',"since":' + JSON.stringify(since) + '}';
}
function createEngine(opts = {}) {
if (!vm.constants || vm.constants.DONT_CONTEXTIFY === undefined) {
throw new Error(`server-run play needs Node 22.8 or newer (vm.constants.DONT_CONTEXTIFY); this is ${process.version}`);
}
const N = Math.max(1, Math.floor(+opts.workers || 2));
const heapMb = Math.max(64, Math.floor(+opts.heapMb || 1536));
const log = typeof opts.log === 'function' ? opts.log : () => {};
const test = opts.test || null;
const W = { root: opts.root, page: opts.page, test, strict: opts.strict !== undefined ? !!opts.strict : process.env.SIM_STRICT === '1', timeouts: opts.timeouts };
let seq = 0, closed = false;
const pool = [], own = new Map(), placed = new Map(), exits = new Map(), gaveUp = new Set();
// A session's realms, counted where they live (placement by realm count).
const count = (w) => [...w.sessions.values()].reduce((a, b) => a + b.size, 0);
function spawn(dedicated = false) {
const w = { worker: null, pending: new Map(), sessions: new Map(), dedicated, dead: false };
w.worker = new Worker(FILE, { workerData: { mpSim: W }, resourceLimits: { maxOldGenerationSizeMb: heapMb } });
w.worker.on('message', (m) => {
if (m.log) { log(m.log[0], m.log[1]); return; }
if (m.fatal) { log('error', 'mp-sim worker could not start: ' + m.fatal); return; }
if (m.ready) return;
const p = w.pending.get(m.id);
if (!p) return;
w.pending.delete(m.id);
// An idle worker does not keep the process alive; one with calls pending does.
if (!w.pending.size) w.worker.unref();
if (m.ok) p.ok(m.reply);
else p.no(Object.assign(new Error(m.error.message), { code: m.error.code }));
});
w.worker.on('error', (e) => log('error', 'mp-sim worker failed: ' + (e && e.message)));
w.worker.on('exit', (c) => died(w, c));
w.worker.unref();
return w;
}
function died(w, c) {
if (w.dead) return;
w.dead = true;
const e = Object.assign(new Error('worker exited'), { code: 'EXIT', exitCode: c });
for (const p of w.pending.values()) p.no(e);
w.pending.clear();
const t = Date.now();
for (const code of w.sessions.keys()) {
placed.delete(code);
if (w.dedicated) { gaveUp.add(code); own.delete(code); log('error', `mp-sim: session ${code} took down its own worker too; not loading it again until a restart`); continue; }
exits.set(code, (exits.get(code) || []).filter((x) => t - x < HOUR).concat(t));
}
w.sessions.clear();
if (!closed) {
if (!closed) log('warn', `mp-sim: a worker exited (code ${c}); its sessions load again from their checkpoints`);
const i = pool.indexOf(w);
if (i >= 0) pool[i] = spawn();
}
}
for (let i = 0; i < N; i++) pool.push(spawn());
// The worker of a session: where it is, else the pool's least loaded one (or one of its own, after two exits).
function workerOf(code, create) {
let w = placed.get(code);
if (w && !w.dead) return w;
if (!create) return null;
if (gaveUp.has(code)) throw Object.assign(new Error(`session ${code} kept failing its worker`), { code: 'GAVE_UP' });
if ((exits.get(code) || []).filter((x) => Date.now() - x < HOUR).length >= 2) {
w = spawn(true); own.set(code, w);
} else w = pool.reduce((a, b) => (count(b) < count(a) ? b : a));
placed.set(code, w);
return w;
}
function send(w, m) {
if (closed) return Promise.reject(Object.assign(new Error('engine closed'), { code: 'CLOSED' }));
if (!w || w.dead) return Promise.reject(Object.assign(new Error('worker exited'), { code: 'EXIT' }));
const id = ++seq;
return new Promise((ok, no) => { if (!w.pending.size) w.worker.ref(); w.pending.set(id, { ok, no }); w.worker.postMessage({ id, ...m }); });
}
const noRealm = (code, nid) => Promise.reject(Object.assign(new Error(`no realm ${code}/${nid}`), { code: 'NO_REALM' }));
const to = (code, nid, m) => { const w = workerOf(code, false); return w ? send(w, { code, nid, ...m }) : noRealm(code, nid); };
const mine = (w, code) => { let s = w.sessions.get(code); if (!s) w.sessions.set(code, s = new Set()); return s; };
const engine = {
simId: null,
load(code, nets) {
if (!Array.isArray(nets) || !nets.length) return Promise.reject(new Error('load: give the networks to load'));
let w;
try { w = workerOf(code, true); } catch (e) { return Promise.reject(e); }
return send(w, { t: 'load', code, nets }).then((r) => {
const s = mine(w, code);
for (const l of r) if (!l.err) s.add(l.nid); else s.delete(l.nid);
return r;
});
},
day(code, { d, speed = 1, last = 1, alone = [], nets = [], mute = false } = {}) {
const w = workerOf(code, false);
if (!w) return Promise.reject(Object.assign(new Error(`no session ${code} here`), { code: 'NO_REALM' }));
return send(w, { t: 'day', code, d, speed, last, alone, nets, mute });
},
act(code, nid, msg) { return to(code, nid, { t: 'act', msg }); },
ask(code, nid, msg) { return to(code, nid, { t: 'ask', msg }); },
pull(code, nid, { snap = false, views } = {}) { return to(code, nid, { t: 'pull', snap: !!snap, ...(views ? { views } : {}) }); },
sub(code, nid, { views = [] } = {}) { return to(code, nid, { t: 'sub', views }); },
prefs(code, nid, { prefs } = {}) { return to(code, nid, { t: 'prefs', prefs }).then(() => undefined); },
greet(code, nid) { return to(code, nid, { t: 'greet' }); },
save(code, nid) { return to(code, nid, { t: 'save' }); },
card(code, nid) { return to(code, nid, { t: 'card' }); },
unload(code, nid) {
const w = workerOf(code, false);
if (!w) return Promise.resolve({ t: 'unloaded', code, ...(nid !== undefined ? { nid } : {}) });
return send(w, { t: 'unload', code, ...(nid !== undefined ? { nid } : {}) }).then((r) => {
const s = w.sessions.get(code);
if (s) { if (nid === undefined) s.clear(); else s.delete(nid); if (!s.size) { w.sessions.delete(code); placed.delete(code); } }
if (w.dedicated && !w.sessions.size) { own.delete(code); w.dead = true; w.worker.terminate(); }
return r;
});
},
stats() {
const ws = [...pool, ...own.values()].filter((w) => !w.dead);
return { workers: ws.length, realms: ws.reduce((a, w) => a + count(w), 0), sessions: ws.reduce((a, w) => a + w.sessions.size, 0) };
},
close() {
closed = true;
const ws = [...pool, ...own.values()];
return Promise.all(ws.map((w) => (w.dead ? null : w.worker.terminate()))).then(() => undefined);
},
snapWith,
};
if (test) {
engine.eval = (code, nid, src, ms) => to(code, nid, { t: 'eval', src: String(src), ...(ms ? { ms } : {}) });
engine.mem = () => Promise.all(pool.filter((w) => !w.dead).map((w) => send(w, { t: 'mem' })));
engine.kill = (i = 0) => { const w = pool[i]; if (w && !w.dead) w.worker.terminate(); };
engine.workerOf = (code) => pool.indexOf(placed.get(code));
}
{ const page = require('./page'); engine.simId = page.simId(page.assemble({ src: path.join(opts.root || page.ROOT, 'src'), mp: true }).html); }
return engine;
}
module.exports = { createEngine, snapWith };