- docs/plans/server-run.md: the server-run design, which lived only outside
the repo, with a status at the top (T1 and T2 merged, T3, T4 and the
integration waiting for the go; the four answers; the questions still
open) and a section on what building T2 changed: a checkpoint needs more
than route memory (rt carries pt, sl, ord and rk), who a worker exit
counts against, the budgets now over, the contract additions, the open
path-window question and what the shared calendar changes. The answers
are marked where sections 0, 1 and 14 ask; probes are named, not pathed.
- docs/plans/next-session.md: rewritten to match main at 3e30621: what is
built, the answers, the holds, the nine open questions, what is in
progress and next, and only the limitations that still hold.
- docs/ARCHITECTURE.md: a table of the fork's own files (build standard,
dark mode, IPv4, IPv6-only, express fiber, satellites, multiplayer,
server-run play, the session server and the engine), kept apart from
upstream's tables so merges stay clean.
- docs/MULTIPLAYER.md: server-run play is being built and not switched on.
222 lines
14 KiB
Markdown
222 lines
14 KiB
Markdown
# Multiplayer (self-hosted copies)
|
||
|
||
The game at [seriesoftub.es](https://seriesoftub.es) is hosted on GitHub Pages and nowhere else
|
||
([ADR 0001](adr/0001-github-pages-only.md)), so it has no server and no multiplayer. This page is for a copy you host
|
||
yourself: there, a small session server that ships with the source (`scripts/mp-server.js`) lets players build their
|
||
networks side by side.
|
||
|
||
A plain `npm run build` is unchanged: the page has no multiplayer door, no chip and makes no request for it. Multiplayer
|
||
is switched on only when the build is told where the session server lives.
|
||
|
||
## What it does (phase 1)
|
||
|
||
- **Sessions by code.** One player starts a session from the title screen (Multiplayer: their network's name, home city
|
||
and rules preset, optionally a session name) and gets a six-character code such as `K7Q-2XD`. Friends join with that
|
||
code. Codes use Crockford's base32 (no I, L, O or U); they are case-insensitive, and the dash is optional.
|
||
- **One world.** Everyone in a session plays a Normal game with the creator's rules preset (Best Effort, Change Freeze,
|
||
On-call or BGP at 3 a.m.) and the session's random seed, so day 0 is the same world for all. The session hands each
|
||
player an AS number from 200000–214999 that nobody else in the session and no network on the map uses. Each network
|
||
is still simulated in its own player's browser; from the first different click on, the worlds drift apart.
|
||
- **Saves on the server.** Each network's save goes to the server on a new game day, at most every 15 seconds (gzipped
|
||
when it is big), and when the page is hidden or closed if anything changed since the last push (a paused game's
|
||
building too). Opening the network on any device fetches the server's save first and loads it if another device
|
||
played on.
|
||
- **Seeing each other.** Every push carries a small public *card*: day, tier, POPs, links between them (with capacity),
|
||
exchanges, transit providers, peering policy, reach, peak and served traffic, customers, revenue per month, cash as an
|
||
order of magnitude, and a score. The session panel (the 🌐 chip in the top bar, or the menu) shows the leaderboard
|
||
(Tier 1 first, then Tier 2, then traffic served), a feed (joins, leaves, Tier 2, Tier 1 and who got there first) and a
|
||
chat. While the panel is open it refreshes every 5 seconds; otherwise the roster rides along with each push.
|
||
- **Moving to another device.** The panel's "Move to another device" holds a link `https://your.host/#mp=CODE.pid.token`.
|
||
The part after `#` never reaches any server. Opening it (or copying the link) first sends this device's latest save,
|
||
and opening the link loads the network into that browser. Treat it like a password: anyone with it can play as you.
|
||
A browser keeps the link's token only once the server has accepted it, so a wrong or mistyped link never replaces the
|
||
one it already has; when the server cannot be reached, the link stays in the address bar for a reload to try again.
|
||
- **Offline.** The game never waits for the server. When it cannot be reached, the game says so once, keeps playing and
|
||
tries again later (30 s, then 1, 2, 4, at most 5 minutes apart; the open session panel keeps looking every 5 seconds
|
||
without stretching that wait).
|
||
|
||
Not yet (phase 2): peering, transit and competition between players inside the simulation. The card already carries
|
||
what that will need (POPs, links, exchanges, transit, peering policy, reach).
|
||
|
||
## Being built: the server runs the game
|
||
|
||
Work has started on server-run play. In a multiplayer build, the session server will run every network's days, so all
|
||
players of a session share one calendar and one speed, and the page will only show the network and send the player's
|
||
clicks. The engine that runs networks on the server is in the source (`scripts/lib/mp-sim-*.js`), but nothing uses it
|
||
yet and it is not switched on: sessions work exactly as this page describes, with each network simulated in its own
|
||
player's browser. The design and how far it has got are in [plans/server-run.md](plans/server-run.md).
|
||
|
||
## Build
|
||
|
||
```bash
|
||
MP_API=/api/mp npm run build # _site/ with multiplayer on; the page talks to /api/mp on its own origin
|
||
```
|
||
|
||
`MP_API` must be a path on the same origin (letters, digits, `/`, `_`, `-`; no trailing slash, and not starting with
|
||
`//`, which a browser reads as another host). Anything else fails the build. Without it, or empty, the build is the
|
||
plain one.
|
||
|
||
## Run the server
|
||
|
||
```bash
|
||
node scripts/mp-server.js --site _site --data /var/lib/tubes-mp --port 8090
|
||
# or: npm run mp -- --site _site --data /var/lib/tubes-mp
|
||
```
|
||
|
||
One process serves the API under `/api/mp` and, with `--site`, the built game too (like GitHub Pages would: `/` is
|
||
`index.html`, a missing file gets `404.html`, nothing outside the folder is ever served). It binds `127.0.0.1` by default:
|
||
put a reverse proxy with HTTPS in front. Plain `node:http`, no dependencies beyond Node 20 or newer.
|
||
|
||
| Option | Default | |
|
||
|---|---|---|
|
||
| `--port N` | 8090 (or `$PORT`) | port to listen on |
|
||
| `--host H` | 127.0.0.1 | address to bind |
|
||
| `--data DIR` | `./mp-data` | sessions and saves; created if missing |
|
||
| `--site DIR` | none | also serve the built game from DIR |
|
||
| `--base PATH` | `/api/mp` | the API's path; must match `MP_API` |
|
||
| `--prune-days N` | 60 | sessions nobody touched for N days are deleted, at startup and daily |
|
||
| `--max-sessions N` | 1000 | sessions on this server |
|
||
| `--max-players N` | 12 | players per session (at most 12) |
|
||
| `--create-per-hour N` | 10 | new sessions one address may start per hour |
|
||
| `--sessions-per-hour N` | 100 | new sessions per hour on the whole server, from all addresses together |
|
||
| `--trust-proxy` | off | take the client's address from `X-Forwarded-For` (only behind your own proxy) |
|
||
| `--verbose` | off | one log line per API request (never tokens) |
|
||
|
||
Behind a reverse proxy every request comes from the proxy's address, so the per-address limits only mean something with
|
||
`--trust-proxy` (the server then uses the last address in `X-Forwarded-For`, the one your proxy added); without it, all
|
||
players share one address's limits, such as 4 save uploads at once. An IPv6 address counts as its /64, since one phone
|
||
or home usually has a whole /64 to itself.
|
||
|
||
To try it locally without a build: `npm run dev -- --mp` runs the API inside the dev server (sessions in `mp-data/`) and
|
||
turns multiplayer on in the dev page. Open a second window (a private one counts as another browser) to be the second
|
||
player.
|
||
|
||
### nginx
|
||
|
||
```nginx
|
||
server {
|
||
listen 443 ssl http2;
|
||
server_name tubes.example.org;
|
||
ssl_certificate /etc/letsencrypt/live/tubes.example.org/fullchain.pem;
|
||
ssl_certificate_key /etc/letsencrypt/live/tubes.example.org/privkey.pem;
|
||
|
||
client_max_body_size 5m; # a save is at most 4 MB
|
||
|
||
location / {
|
||
proxy_pass http://127.0.0.1:8090; # the game and /api/mp, from the one process
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
}
|
||
}
|
||
```
|
||
|
||
Run the server with `--site /srv/tubes/_site --data /var/lib/tubes-mp --trust-proxy`.
|
||
|
||
### Caddy
|
||
|
||
```caddy
|
||
tubes.example.org {
|
||
request_body {
|
||
max_size 5MB
|
||
}
|
||
reverse_proxy 127.0.0.1:8090
|
||
}
|
||
```
|
||
|
||
Caddy sets `X-Forwarded-For` itself and gets the certificate on its own; run the server with `--trust-proxy` as above.
|
||
|
||
### systemd
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=Series of Tubes multiplayer
|
||
After=network.target
|
||
|
||
[Service]
|
||
ExecStart=/usr/bin/node /srv/tubes/scripts/mp-server.js --site /srv/tubes/_site --data /var/lib/tubes-mp --trust-proxy
|
||
User=tubes
|
||
Restart=on-failure
|
||
StateDirectory=tubes-mp
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
## Backups
|
||
|
||
Everything is in the data folder: `<CODE>.json` per session (settings, players, cards, feed, and each save's number) and
|
||
`<CODE>.<pid>.save` per player's save (its text exactly as the game sent it). Every write goes to a temporary file that is then renamed over the old one, so a
|
||
copy of the folder (`rsync`, a snapshot, `tar`) taken while the server runs holds whole files. Restore by stopping the
|
||
server, putting the files back and starting it again. A player's token is stored only as its SHA-256, so a backup cannot
|
||
be used to play as anyone.
|
||
|
||
## Limits
|
||
|
||
- 12 players per session, 1000 sessions per server (configurable).
|
||
- A save is at most 4 MB of text (the game gzips anything over 32 kB: the big test network, 236 kB of JSON, goes as about 50 kB). Other
|
||
requests are at most 16 kB.
|
||
- Chat lines: at most 200 characters, one a second and six per 30 seconds per player. The feed keeps the last 100 lines.
|
||
- Per address and hour (an IPv6 address counts as its /64): 10 new sessions, 60 joins, 120 wrong codes (after that,
|
||
every code is refused for a while from there, so guessing codes gets nowhere; a player who sends their own session's
|
||
code with its token is never refused for it). At most 100 new sessions an hour on the whole server.
|
||
- Saves and cards: one save upload at a time per player (another one meanwhile gets 429), at most 4 at once from one
|
||
address and 16 on the whole server (503 with `busy: true` beyond that; the game then tries again a little later,
|
||
quietly), and per player at most 20 saves and 60 cards a minute (the game sends one every 15 seconds at most). A
|
||
save's body must arrive within two minutes. A card alone is written to disk within two seconds, together with any
|
||
others that came in meanwhile.
|
||
- Network names: at most 32 characters; control characters, `<`, `>`, `&`, quotes and backticks are removed. The page
|
||
escapes everything other players wrote anyway.
|
||
- A page that is being hidden or closed can only send 64 KiB (the browser's limit for such requests, counted in bytes),
|
||
so a network whose save does not fit sends its card alone then; its save goes with the next push from that device.
|
||
|
||
## How saves stay in order
|
||
|
||
Each network in a session carries a save number (`S.mp.seq`). When a save is due to go to the server, the game picks
|
||
the next number before it writes the save, so the save kept in the browser and the one sent carry the same number;
|
||
reopening the network then finds nothing newer on the server unless another device pushed one. The server refuses a
|
||
number that is not newer than its own (HTTP 409, with its number), and only counts a save once it is on disk. Then the
|
||
player is told once that the network moved on elsewhere and chooses: load the newer save from the server (only one at
|
||
least as new as the server said it had: never an older one), or keep this one (at the end of the current game day it
|
||
goes with `force` and a number past the server's).
|
||
|
||
The token is never part of the save: it lives in the browser's storage under `sot-mp:<code>:<pid>`. An exported or
|
||
imported save keeps its session but cannot talk to the server without the token (the game says it is no longer in its
|
||
session and plays on alone).
|
||
|
||
## API
|
||
|
||
JSON in and out, same origin, no cookies, no CORS headers. Every call but starting and joining needs
|
||
`Authorization: Bearer <pid>.<token>`. Errors are `{"error": "…"}` with a fitting status (400, 401, 404, 405, 409, 413,
|
||
415, 429, 503).
|
||
|
||
| Call | Body | Answer |
|
||
|---|---|---|
|
||
| `POST /api/mp/sessions` | `{net:{name,home}, preset, title?, avoid?}` | `201 {code, pid, token, seed, asn, preset, color, title}` |
|
||
| `POST /api/mp/sessions/:code/join` | `{net:{name,home}, avoid?}` | the same, for the joiner (`409 {full:true}` when full) |
|
||
| `GET /api/mp/sessions/:code` | | `{code, title, preset, seed, created, players:[{pid, name, asn, color, card, seen, joined}], feed}` |
|
||
| `PUT /api/mp/sessions/:code/card` | `{card}` | the roster |
|
||
| `PUT /api/mp/sessions/:code/save` | `{seq, at, data, card?, force?}` | `{ok, seq, at, roster}`, or `409 {seq, at}`; `429` while another one is on its way, `503` when the server is busy |
|
||
| `GET /api/mp/sessions/:code/save` | | `{seq, at, data}` (your own save only) |
|
||
| `POST /api/mp/sessions/:code/feed` | `{text}` | the roster |
|
||
| `DELETE /api/mp/sessions/:code/me` | | `{ok}`; the session goes when its last player leaves |
|
||
|
||
`avoid` lists AS numbers the server must not hand out (the game sends the map's own networks in that range). `preset` is
|
||
one of `soft`, `freeze`, `oncall`, `bgp`. `data` is the save as the game writes it: JSON text, or `gz:` and base64.
|
||
|
||
The card, as the server keeps it (unknown fields are dropped, numbers clamped, ids checked for shape; the page checks
|
||
the ids against its own catalogs again): `day`, `tier` (`t1`, `t2`, `t3`), `t1`, `lvl`, `pops` (city ids), `links`
|
||
(`[a, b, Gbps]`, up to 300), `ix`, `transit` (network ids), `ppol` (`open`, `selective`, `closed`), `reach` (0–1),
|
||
`peak`, `served` (Gbps), `cust`, `rev` (per month), `cashB` (−1 in debt, else the power of ten of the cash), `score`.
|
||
Cards are what each player's browser says: nothing on the server checks that a network really is that big.
|
||
|
||
## Code
|
||
|
||
- Server: `scripts/lib/mp.js` (the API and storage), `scripts/mp-server.js` (the command line and the static site).
|
||
- Game: `src/game/60-features/89-multiplayer.js` (state, sync, the card, the `#mp=` link),
|
||
`src/game/95-modals/48-multiplayer.js` (door, sheets, panel, chip), `src/game/90-actions/92-multiplayer.js`
|
||
(actions), `src/styles/41-multiplayer.css`. The build stamps `MP_API` into `src/game/00-core/20-analytics.js`.
|
||
- Tests: `tests/mp-server.test.js`, `tests/multiplayer.test.js`.
|
||
- Server-run play (being built, not switched on): the act tables `scripts/lib/mp-sim-acts.js`, the engine
|
||
`scripts/lib/mp-sim-engine.js`, `mp-sim-worker.js` and `mp-sim-realm.js`, the page's side `src/game/98-server-play/`
|
||
(a comment so far), and the tests `tests/mp-sim-*.test.js` and `tests/server-sim-prereqs.test.js`.
|