seriesoftubes/docs/MULTIPLAYER.md
jvanvik 95aa9b6532 Multiplayer for self-hosted copies: sessions, server-side saves, a leaderboard
Off unless the page is built with MP_API set (MP_API=/api/mp npm run
build): a plain build has no door, no chip and makes no requests. The
session server is scripts/mp-server.js (npm run mp), plain node:http,
which can also serve the built site so one process sits behind the
reverse proxy; docs/MULTIPLAYER.md has the nginx and Caddy setups.

A session has a six-character code, one seed and one rules preset, so
every player starts in the same world, and up to twelve players with
their own AS numbers. Saves go to the server on a new game day at most
every 15 s and when the page hides, ordered by a sequence number (409 on
a stale push, and the player picks which one to keep). Tokens live in
localStorage, never in the save; the server keeps only their hash. A
#mp= link moves a network to another device. The session panel shows
the leaderboard, the feed and a short chat; each player's public card
(tier, POPs, links, exchanges, transit, peak, customers, score) carries
what in-game peering between players will need next.

Server limits: body sizes, one upload per player, uploads per address
and in total, sessions per hour, wrong-code lockout; addresses counted
per IPv6 /64, X-Forwarded-For only with --trust-proxy.
2026-11-07 00:42:39 +01:00

12 KiB
Raw Blame History

Multiplayer (self-hosted copies)

The game at seriesoftub.es is hosted on GitHub Pages and nowhere else (ADR 0001), so it has no server and no multiplayer. This page is for a copy you host yourself: there, a small session server that ships with the source (scripts/mp-server.js) lets players build their networks side by side.

A plain npm run build is unchanged: the page has no multiplayer door, no chip and makes no request for it. Multiplayer is switched on only when the build is told where the session server lives.

What it does (phase 1)

  • Sessions by code. One player starts a session from the title screen (Multiplayer: their network's name, home city and rules preset, optionally a session name) and gets a six-character code such as K7Q-2XD. Friends join with that code. Codes use Crockford's base32 (no I, L, O or U); they are case-insensitive, and the dash is optional.
  • One world. Everyone in a session plays a Normal game with the creator's rules preset (Best Effort, Change Freeze, On-call or BGP at 3 a.m.) and the session's random seed, so day 0 is the same world for all. The session hands each player an AS number from 200000–214999 that nobody else in the session and no network on the map uses. Each network is still simulated in its own player's browser; from the first different click on, the worlds drift apart.
  • Saves on the server. Each network's save goes to the server on a new game day, at most every 15 seconds (gzipped when it is big), and when the page is hidden or closed if anything changed since the last push (a paused game's building too). Opening the network on any device fetches the server's save first and loads it if another device played on.
  • Seeing each other. Every push carries a small public card: day, tier, POPs, links between them (with capacity), exchanges, transit providers, peering policy, reach, peak and served traffic, customers, revenue per month, cash as an order of magnitude, and a score. The session panel (the 🌐 chip in the top bar, or the menu) shows the leaderboard (Tier 1 first, then Tier 2, then traffic served), a feed (joins, leaves, Tier 2, Tier 1 and who got there first) and a chat. While the panel is open it refreshes every 5 seconds; otherwise the roster rides along with each push.
  • Moving to another device. The panel's "Move to another device" holds a link https://your.host/#mp=CODE.pid.token. The part after # never reaches any server. Opening it (or copying the link) first sends this device's latest save, and opening the link loads the network into that browser. Treat it like a password: anyone with it can play as you. A browser keeps the link's token only once the server has accepted it, so a wrong or mistyped link never replaces the one it already has; when the server cannot be reached, the link stays in the address bar for a reload to try again.
  • Offline. The game never waits for the server. When it cannot be reached, the game says so once, keeps playing and tries again later (30 s, then 1, 2, 4, at most 5 minutes apart; the open session panel keeps looking every 5 seconds without stretching that wait).

Not yet (phase 2): peering, transit and competition between players inside the simulation. The card already carries what that will need (POPs, links, exchanges, transit, peering policy, reach).

Build

MP_API=/api/mp npm run build        # _site/ with multiplayer on; the page talks to /api/mp on its own origin

MP_API must be a path on the same origin (letters, digits, /, _, -; no trailing slash, and not starting with //, which a browser reads as another host). Anything else fails the build. Without it, or empty, the build is the plain one.

Run the server

node scripts/mp-server.js --site _site --data /var/lib/tubes-mp --port 8090
# or: npm run mp -- --site _site --data /var/lib/tubes-mp

One process serves the API under /api/mp and, with --site, the built game too (like GitHub Pages would: / is index.html, a missing file gets 404.html, nothing outside the folder is ever served). It binds 127.0.0.1 by default: put a reverse proxy with HTTPS in front. Plain node:http, no dependencies beyond Node 20 or newer.

Option Default
--port N 8090 (or $PORT) port to listen on
--host H 127.0.0.1 address to bind
--data DIR ./mp-data sessions and saves; created if missing
--site DIR none also serve the built game from DIR
--base PATH /api/mp the API's path; must match MP_API
--prune-days N 60 sessions nobody touched for N days are deleted, at startup and daily
--max-sessions N 1000 sessions on this server
--max-players N 12 players per session (at most 12)
--create-per-hour N 10 new sessions one address may start per hour
--sessions-per-hour N 100 new sessions per hour on the whole server, from all addresses together
--trust-proxy off take the client's address from X-Forwarded-For (only behind your own proxy)
--verbose off one log line per API request (never tokens)

Behind a reverse proxy every request comes from the proxy's address, so the per-address limits only mean something with --trust-proxy (the server then uses the last address in X-Forwarded-For, the one your proxy added); without it, all players share one address's limits, such as 4 save uploads at once. An IPv6 address counts as its /64, since one phone or home usually has a whole /64 to itself.

To try it locally without a build: npm run dev -- --mp runs the API inside the dev server (sessions in mp-data/) and turns multiplayer on in the dev page. Open a second window (a private one counts as another browser) to be the second player.

nginx

server {
    listen 443 ssl http2;
    server_name tubes.example.org;
    ssl_certificate     /etc/letsencrypt/live/tubes.example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/tubes.example.org/privkey.pem;

    client_max_body_size 5m;            # a save is at most 4 MB

    location / {
        proxy_pass http://127.0.0.1:8090;   # the game and /api/mp, from the one process
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Run the server with --site /srv/tubes/_site --data /var/lib/tubes-mp --trust-proxy.

Caddy

tubes.example.org {
    request_body {
        max_size 5MB
    }
    reverse_proxy 127.0.0.1:8090
}

Caddy sets X-Forwarded-For itself and gets the certificate on its own; run the server with --trust-proxy as above.

systemd

[Unit]
Description=Series of Tubes multiplayer
After=network.target

[Service]
ExecStart=/usr/bin/node /srv/tubes/scripts/mp-server.js --site /srv/tubes/_site --data /var/lib/tubes-mp --trust-proxy
User=tubes
Restart=on-failure
StateDirectory=tubes-mp

[Install]
WantedBy=multi-user.target

Backups

Everything is in the data folder: <CODE>.json per session (settings, players, cards, feed, and each save's number) and <CODE>.<pid>.save per player's save (its text exactly as the game sent it). Every write goes to a temporary file that is then renamed over the old one, so a copy of the folder (rsync, a snapshot, tar) taken while the server runs holds whole files. Restore by stopping the server, putting the files back and starting it again. A player's token is stored only as its SHA-256, so a backup cannot be used to play as anyone.

Limits

  • 12 players per session, 1000 sessions per server (configurable).
  • A save is at most 4 MB of text (the game gzips anything over 32 kB: the big test network, 236 kB of JSON, goes as about 50 kB). Other requests are at most 16 kB.
  • Chat lines: at most 200 characters, one a second and six per 30 seconds per player. The feed keeps the last 100 lines.
  • Per address and hour (an IPv6 address counts as its /64): 10 new sessions, 60 joins, 120 wrong codes (after that, every code is refused for a while from there, so guessing codes gets nowhere; a player who sends their own session's code with its token is never refused for it). At most 100 new sessions an hour on the whole server.
  • Saves and cards: one save upload at a time per player (another one meanwhile gets 429), at most 4 at once from one address and 16 on the whole server (503 with busy: true beyond that; the game then tries again a little later, quietly), and per player at most 20 saves and 60 cards a minute (the game sends one every 15 seconds at most). A save's body must arrive within two minutes. A card alone is written to disk within two seconds, together with any others that came in meanwhile.
  • Network names: at most 32 characters; control characters, <, >, &, quotes and backticks are removed. The page escapes everything other players wrote anyway.
  • A page that is being hidden or closed can only send 64 KiB (the browser's limit for such requests, counted in bytes), so a network whose save does not fit sends its card alone then; its save goes with the next push from that device.

How saves stay in order

Each network in a session carries a save number (S.mp.seq). When a save is due to go to the server, the game picks the next number before it writes the save, so the save kept in the browser and the one sent carry the same number; reopening the network then finds nothing newer on the server unless another device pushed one. The server refuses a number that is not newer than its own (HTTP 409, with its number), and only counts a save once it is on disk. Then the player is told once that the network moved on elsewhere and chooses: load the newer save from the server (only one at least as new as the server said it had: never an older one), or keep this one (at the end of the current game day it goes with force and a number past the server's).

The token is never part of the save: it lives in the browser's storage under sot-mp:<code>:<pid>. An exported or imported save keeps its session but cannot talk to the server without the token (the game says it is no longer in its session and plays on alone).

API

JSON in and out, same origin, no cookies, no CORS headers. Every call but starting and joining needs Authorization: Bearer <pid>.<token>. Errors are {"error": "…"} with a fitting status (400, 401, 404, 405, 409, 413, 415, 429, 503).

Call Body Answer
POST /api/mp/sessions {net:{name,home}, preset, title?, avoid?} 201 {code, pid, token, seed, asn, preset, color, title}
POST /api/mp/sessions/:code/join {net:{name,home}, avoid?} the same, for the joiner (409 {full:true} when full)
GET /api/mp/sessions/:code {code, title, preset, seed, created, players:[{pid, name, asn, color, card, seen, joined}], feed}
PUT /api/mp/sessions/:code/card {card} the roster
PUT /api/mp/sessions/:code/save {seq, at, data, card?, force?} {ok, seq, at, roster}, or 409 {seq, at}; 429 while another one is on its way, 503 when the server is busy
GET /api/mp/sessions/:code/save {seq, at, data} (your own save only)
POST /api/mp/sessions/:code/feed {text} the roster
DELETE /api/mp/sessions/:code/me {ok}; the session goes when its last player leaves

avoid lists AS numbers the server must not hand out (the game sends the map's own networks in that range). preset is one of soft, freeze, oncall, bgp. data is the save as the game writes it: JSON text, or gz: and base64.

The card, as the server keeps it (unknown fields are dropped, numbers clamped, ids checked for shape; the page checks the ids against its own catalogs again): day, tier (t1, t2, t3), t1, lvl, pops (city ids), links ([a, b, Gbps], up to 300), ix, transit (network ids), ppol (open, selective, closed), reach (0–1), peak, served (Gbps), cust, rev (per month), cashB (−1 in debt, else the power of ten of the cash), score. Cards are what each player's browser says: nothing on the server checks that a network really is that big.

Code

  • Server: scripts/lib/mp.js (the API and storage), scripts/mp-server.js (the command line and the static site).
  • Game: src/game/60-features/89-multiplayer.js (state, sync, the card, the #mp= link), src/game/95-modals/48-multiplayer.js (door, sheets, panel, chip), src/game/90-actions/92-multiplayer.js (actions), src/styles/41-multiplayer.css. The build stamps MP_API into src/game/00-core/20-analytics.js.
  • Tests: tests/mp-server.test.js, tests/multiplayer.test.js.