Server-run play, part 2: the engine
The engine that runs every network of a server session (server-sim design,
section 4): each network in its own vm realm, realms in a small pool of
worker threads, a promise API for the session host. Nothing uses it yet;
the plain build and the game are unchanged (no file under src/).
- scripts/lib/mp-sim-realm.js: one network in a DONT_CONTEXTIFY realm
running the plain page's game (never the MP page's client folder), with
recording DOM stand-ins, no-op timers that count, and the srv... calls:
load from a checkpoint (route memory seeded, then routed), begin a new
network, marks and restores, the day as a page runs it, the action
pipeline with drafts, arming, the sanitize gate and rollback, what-ifs
that put S, R, the dice and uidN back, pulls with top-level key diffs,
slim R and flow records on demand, fx captured and bounded, saves, cards.
- scripts/lib/mp-sim-worker.js: realms per session in one worker, days one
network at a time with other messages between them, loads and unloads
waiting for a session's day, time limits; a throw goes back to the mark
in place, a timeout gets a fresh realm (new epoch) carrying the old R, a
day is retried once and the network broken after a second failure.
- scripts/lib/mp-sim-engine.js: createEngine() with the fake engine's API,
placement by realm count, every pending call rejected when its worker
exits, a worker of its own after two exits in an hour; snapWith() joins
the host's clock, roster and since to a realm's snapshot.
- Contract: a pulled snapshot is the realm's half (the host adds clock,
roster, since); haptic fx carry milliseconds; the fake engine follows,
with close() and snapWith(). The button crawl moved to tests/lib.
- Tests: realm vs jsdom byte for byte over 60 scenic big-network days with
actions; restore; retry after a throw or a timeout; realm order; a timer
audit and the gate over a sweep of sent buttons (SIM_LONG=1: a year and
every button); captures, rollbacks, asks, selection reports, ordering,
bounds, pollution, worker exits; the act tables against the game.
The gate found two keys the game writes in a shape a reload repairs (buyTE's
S.te, mkLeave's S.monkey): game fixes for the integrator, kept visible as a
todo test and a list in the audit.
Rebased onto the 3.3.0 merge (61e31c5): the engine itself needed no change. A
realm still routes in the game's own code (no route worker, no saves), the
route cache that now keeps list orders from day to day leaves a checkpoint and a
retry the same, and the capacity market and moved customers play in a realm as
on a page, under the strict gate. The tests needed three things: an old save's
first days earn the new traffic badges, which the unwatched network's snapshot
keeps; the restore test compares JSON text with sorted keys instead of a
deep-equal of two networks (a diff of gigabytes when they differ); and the crawl
opens the windows that move a customer, with a new test playing the market's and
the path moves' acts, 40 days of sales and a checkpoint against the page.
This commit is contained in:
parent
61e31c53d3
commit
065c922287
12 changed files with 1906 additions and 29 deletions
182
scripts/lib/mp-sim-engine.js
Normal file
182
scripts/lib/mp-sim-engine.js
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
// The server-run engine (the server-sim design, sections 4.2-4.3): a small pool of worker threads (mp-sim-worker.js),
|
||||
// each holding the vm realms (mp-sim-realm.js) of the sessions placed on it, behind a promise API. The session host
|
||||
// (scripts/lib/mp-sim.js) keeps the clock, presence and votes, and asks this for days, actions, what-ifs and pulls.
|
||||
// tests/lib/mp-sim-fake-engine.js has the same API with canned answers.
|
||||
//
|
||||
// const engine = createEngine({workers: 2, heapMb: 1536, root, log});
|
||||
// engine.load(code, [{nid, env?, begin?, lang, prefs, rng?}]) → [{t:'loaded', nid, epoch, day, lost, saved?} |
|
||||
// {t:'loaded', nid, err}], one per network, in order. env: the checkpoint envelope {s, uidN, rt, last,
|
||||
// day} (s: the save text); begin: {name, home, preset, asn, mp, tut} builds a new network, checkpointed
|
||||
// at once (saved). A network whose save does not load (or that cannot begin) answers err: broken (4.7).
|
||||
// engine.day(code, {d, speed, last, alone:[nid], nets:[nid], mute}) → {t:'dayDone', code, d, nets:{[nid]: {day, lost,
|
||||
// pause?, err?, retried?, ms}}}. Each network in sorted order; one that failed twice answers err and
|
||||
// its day as it was, and is broken from then on (no more days; actions answer 'stopped').
|
||||
// engine.act(code, nid, msg) → {t:'actDone', nid, epoch, rid, outcome, why?, v, upd} (upd: the update with the result,
|
||||
// version v; null when stopped) | {t:'err', code, nid, rid, what:'timeout'|'throw', msg}
|
||||
// engine.ask(code, nid, msg) → {t:'askDone', nid, rid, modal?, ui?, err?} | {t:'err', …}
|
||||
// engine.pull(code, nid, {snap, views}) → {t:'pulled', nid, epoch, v, upd?, snap?, subs?}. snap is the realm's half
|
||||
// of a snapshot: snapWith(snap, {clock, roster, since}) makes the stream event.
|
||||
// engine.sub(code, nid, {views}) → {t:'subbed', nid, epoch, v, subs}; engine.greet(code, nid) → a pulled
|
||||
// engine.prefs(code, nid, {prefs}) → undefined; engine.save(code, nid) → {t:'saved', nid, txt, uidN, rt, day, lost}
|
||||
// engine.card(code, nid) → {t:'card', nid, card}; engine.unload(code, nid?) → {t:'unloaded', code, nid?}
|
||||
// engine.stats() → {workers, realms, sessions} (not a promise); engine.close() → stops the workers
|
||||
// engine.simId the build id this engine's game belongs to (SIM_ID of the multiplayer page from the same src/)
|
||||
//
|
||||
// Rejections: a call for a network that is not loaded (code 'NO_REALM'), a malformed call, and every call pending on
|
||||
// a worker that exits (out of memory, a crash): Error('worker exited'), code 'EXIT'; that worker's sessions are gone and
|
||||
// the host loads them again (4.7). A session whose worker exited twice within an hour gets a worker of its own; when that
|
||||
// one exits too, its loads reject with code 'GAVE_UP' until the engine is made again (the server restarts).
|
||||
// Epochs: every realm load mints one (a load, a begin, the fresh realm after a timeout); loaded, pulled, actDone and
|
||||
// subbed carry the realm's current one. Versions are the realm's: only a pull (or an action's own pull) makes one.
|
||||
//
|
||||
// Tests only, with createEngine({test: {seed, now, step} | true}): engine.eval(code, nid, src) runs code in a realm
|
||||
// ({value} | {error} | {timeout, epoch}); engine.mem() reports each worker's heap; engine.kill(i) ends worker i.
|
||||
'use strict';
|
||||
const path = require('node:path');
|
||||
const vm = require('node:vm');
|
||||
const { Worker } = require('node:worker_threads');
|
||||
|
||||
const FILE = path.join(__dirname, 'mp-sim-worker.js');
|
||||
const HOUR = 3600e3;
|
||||
|
||||
/* The stream event of a snapshot: the realm's half (S, r, keep, epoch, v, now) with the host's (clock, roster,
|
||||
since), joined as text: the realm's 270 KB are never parsed again. */
|
||||
function snapWith(snap, { clock, roster, since }) {
|
||||
if (typeof snap !== 'string' || !snap.endsWith('}')) throw new Error('snapWith: not a snapshot');
|
||||
return snap.slice(0, -1) + ',"clock":' + JSON.stringify(clock) + ',"roster":' + JSON.stringify(roster) + ',"since":' + JSON.stringify(since) + '}';
|
||||
}
|
||||
|
||||
function createEngine(opts = {}) {
|
||||
if (!vm.constants || vm.constants.DONT_CONTEXTIFY === undefined) {
|
||||
throw new Error(`server-run play needs Node 22.8 or newer (vm.constants.DONT_CONTEXTIFY); this is ${process.version}`);
|
||||
}
|
||||
const N = Math.max(1, Math.floor(+opts.workers || 2));
|
||||
const heapMb = Math.max(64, Math.floor(+opts.heapMb || 1536));
|
||||
const log = typeof opts.log === 'function' ? opts.log : () => {};
|
||||
const test = opts.test || null;
|
||||
const W = { root: opts.root, page: opts.page, test, strict: opts.strict !== undefined ? !!opts.strict : process.env.SIM_STRICT === '1', timeouts: opts.timeouts };
|
||||
let seq = 0, closed = false;
|
||||
const pool = [], own = new Map(), placed = new Map(), exits = new Map(), gaveUp = new Set();
|
||||
// A session's realms, counted where they live (placement by realm count).
|
||||
const count = (w) => [...w.sessions.values()].reduce((a, b) => a + b.size, 0);
|
||||
|
||||
function spawn(dedicated = false) {
|
||||
const w = { worker: null, pending: new Map(), sessions: new Map(), dedicated, dead: false };
|
||||
w.worker = new Worker(FILE, { workerData: { mpSim: W }, resourceLimits: { maxOldGenerationSizeMb: heapMb } });
|
||||
w.worker.on('message', (m) => {
|
||||
if (m.log) { log(m.log[0], m.log[1]); return; }
|
||||
if (m.fatal) { log('error', 'mp-sim worker could not start: ' + m.fatal); return; }
|
||||
if (m.ready) return;
|
||||
const p = w.pending.get(m.id);
|
||||
if (!p) return;
|
||||
w.pending.delete(m.id);
|
||||
// An idle worker does not keep the process alive; one with calls pending does.
|
||||
if (!w.pending.size) w.worker.unref();
|
||||
if (m.ok) p.ok(m.reply);
|
||||
else p.no(Object.assign(new Error(m.error.message), { code: m.error.code }));
|
||||
});
|
||||
w.worker.on('error', (e) => log('error', 'mp-sim worker failed: ' + (e && e.message)));
|
||||
w.worker.on('exit', (c) => died(w, c));
|
||||
w.worker.unref();
|
||||
return w;
|
||||
}
|
||||
function died(w, c) {
|
||||
if (w.dead) return;
|
||||
w.dead = true;
|
||||
const e = Object.assign(new Error('worker exited'), { code: 'EXIT', exitCode: c });
|
||||
for (const p of w.pending.values()) p.no(e);
|
||||
w.pending.clear();
|
||||
const t = Date.now();
|
||||
for (const code of w.sessions.keys()) {
|
||||
placed.delete(code);
|
||||
if (w.dedicated) { gaveUp.add(code); own.delete(code); log('error', `mp-sim: session ${code} took down its own worker too; not loading it again until a restart`); continue; }
|
||||
exits.set(code, (exits.get(code) || []).filter((x) => t - x < HOUR).concat(t));
|
||||
}
|
||||
w.sessions.clear();
|
||||
if (!closed) {
|
||||
if (!closed) log('warn', `mp-sim: a worker exited (code ${c}); its sessions load again from their checkpoints`);
|
||||
const i = pool.indexOf(w);
|
||||
if (i >= 0) pool[i] = spawn();
|
||||
}
|
||||
}
|
||||
for (let i = 0; i < N; i++) pool.push(spawn());
|
||||
|
||||
// The worker of a session: where it is, else the pool's least loaded one (or one of its own, after two exits).
|
||||
function workerOf(code, create) {
|
||||
let w = placed.get(code);
|
||||
if (w && !w.dead) return w;
|
||||
if (!create) return null;
|
||||
if (gaveUp.has(code)) throw Object.assign(new Error(`session ${code} kept failing its worker`), { code: 'GAVE_UP' });
|
||||
if ((exits.get(code) || []).filter((x) => Date.now() - x < HOUR).length >= 2) {
|
||||
w = spawn(true); own.set(code, w);
|
||||
} else w = pool.reduce((a, b) => (count(b) < count(a) ? b : a));
|
||||
placed.set(code, w);
|
||||
return w;
|
||||
}
|
||||
function send(w, m) {
|
||||
if (closed) return Promise.reject(Object.assign(new Error('engine closed'), { code: 'CLOSED' }));
|
||||
if (!w || w.dead) return Promise.reject(Object.assign(new Error('worker exited'), { code: 'EXIT' }));
|
||||
const id = ++seq;
|
||||
return new Promise((ok, no) => { if (!w.pending.size) w.worker.ref(); w.pending.set(id, { ok, no }); w.worker.postMessage({ id, ...m }); });
|
||||
}
|
||||
const noRealm = (code, nid) => Promise.reject(Object.assign(new Error(`no realm ${code}/${nid}`), { code: 'NO_REALM' }));
|
||||
const to = (code, nid, m) => { const w = workerOf(code, false); return w ? send(w, { code, nid, ...m }) : noRealm(code, nid); };
|
||||
const mine = (w, code) => { let s = w.sessions.get(code); if (!s) w.sessions.set(code, s = new Set()); return s; };
|
||||
|
||||
const engine = {
|
||||
simId: null,
|
||||
load(code, nets) {
|
||||
if (!Array.isArray(nets) || !nets.length) return Promise.reject(new Error('load: give the networks to load'));
|
||||
let w;
|
||||
try { w = workerOf(code, true); } catch (e) { return Promise.reject(e); }
|
||||
return send(w, { t: 'load', code, nets }).then((r) => {
|
||||
const s = mine(w, code);
|
||||
for (const l of r) if (!l.err) s.add(l.nid); else s.delete(l.nid);
|
||||
return r;
|
||||
});
|
||||
},
|
||||
day(code, { d, speed = 1, last = 1, alone = [], nets = [], mute = false } = {}) {
|
||||
const w = workerOf(code, false);
|
||||
if (!w) return Promise.reject(Object.assign(new Error(`no session ${code} here`), { code: 'NO_REALM' }));
|
||||
return send(w, { t: 'day', code, d, speed, last, alone, nets, mute });
|
||||
},
|
||||
act(code, nid, msg) { return to(code, nid, { t: 'act', msg }); },
|
||||
ask(code, nid, msg) { return to(code, nid, { t: 'ask', msg }); },
|
||||
pull(code, nid, { snap = false, views } = {}) { return to(code, nid, { t: 'pull', snap: !!snap, ...(views ? { views } : {}) }); },
|
||||
sub(code, nid, { views = [] } = {}) { return to(code, nid, { t: 'sub', views }); },
|
||||
prefs(code, nid, { prefs } = {}) { return to(code, nid, { t: 'prefs', prefs }).then(() => undefined); },
|
||||
greet(code, nid) { return to(code, nid, { t: 'greet' }); },
|
||||
save(code, nid) { return to(code, nid, { t: 'save' }); },
|
||||
card(code, nid) { return to(code, nid, { t: 'card' }); },
|
||||
unload(code, nid) {
|
||||
const w = workerOf(code, false);
|
||||
if (!w) return Promise.resolve({ t: 'unloaded', code, ...(nid !== undefined ? { nid } : {}) });
|
||||
return send(w, { t: 'unload', code, ...(nid !== undefined ? { nid } : {}) }).then((r) => {
|
||||
const s = w.sessions.get(code);
|
||||
if (s) { if (nid === undefined) s.clear(); else s.delete(nid); if (!s.size) { w.sessions.delete(code); placed.delete(code); } }
|
||||
if (w.dedicated && !w.sessions.size) { own.delete(code); w.dead = true; w.worker.terminate(); }
|
||||
return r;
|
||||
});
|
||||
},
|
||||
stats() {
|
||||
const ws = [...pool, ...own.values()].filter((w) => !w.dead);
|
||||
return { workers: ws.length, realms: ws.reduce((a, w) => a + count(w), 0), sessions: ws.reduce((a, w) => a + w.sessions.size, 0) };
|
||||
},
|
||||
close() {
|
||||
closed = true;
|
||||
const ws = [...pool, ...own.values()];
|
||||
return Promise.all(ws.map((w) => (w.dead ? null : w.worker.terminate()))).then(() => undefined);
|
||||
},
|
||||
snapWith,
|
||||
};
|
||||
if (test) {
|
||||
engine.eval = (code, nid, src, ms) => to(code, nid, { t: 'eval', src: String(src), ...(ms ? { ms } : {}) });
|
||||
engine.mem = () => Promise.all(pool.filter((w) => !w.dead).map((w) => send(w, { t: 'mem' })));
|
||||
engine.kill = (i = 0) => { const w = pool[i]; if (w && !w.dead) w.worker.terminate(); };
|
||||
engine.workerOf = (code) => pool.indexOf(placed.get(code));
|
||||
}
|
||||
{ const page = require('./page'); engine.simId = page.simId(page.assemble({ src: path.join(opts.root || page.ROOT, 'src'), mp: true }).html); }
|
||||
return engine;
|
||||
}
|
||||
|
||||
module.exports = { createEngine, snapWith };
|
||||
Loading…
Add table
Add a link
Reference in a new issue