towerdef/docs/DEPLOY.md
Claude a5bd09d505 Phone UI and touch precision overhaul, proxy fixes, Orbital Uplink, four new maps
Phone panels (reviewed + QA'd, matrix of 22 layouts with real taps/bounding boxes):
- selected-tower action bar over the board (Upgrade/Sell/Target/close), info panel
  updated in place so taps are never lost, scrollable palette for 7+ towers,
  overlays with pinned footers (Help "Got it", Play solo, Join always visible),
  tappable room chip to invite, reduced-motion/contrast/forced-colors fixes,
  paused state, partner cursor labels, bigger level pips and owner flags
- touch input rewritten around "pointerdown commits nothing": pinch never builds,
  10px tap slop, palm-proof pinch, long-press pings with any tool armed, nudge
  arrows for precise placement, hotkey repeat guard, proportional wheel zoom
- new e2e: select/sell, info-panel stability, UI states, touch precision;
  camera unit tests

Proxy/server (reproduced against real nginx 1.26 with NPM v2.16 config):
- Origin check works when the proxy drops a non-default port, TRUSTED_PROXIES for
  proxies on global IPv6/CGNAT addresses, service worker no longer serves nginx's
  502 page, DEPLOY.md corrected (NPM default read timeout is 90 s, Advanced-tab
  limits, path-prefix, cookie size, per-IP cap behaviour)

Content:
- Orbital Uplink: 4-level endgame tower (450 + 500/800/1200 credits): piercing
  armour-ignoring beam, blast and lingering slow, ground+air, range 11-14;
  glyph, beam effect, sound, help text, unit tests
- Maps: The Canyon, Archipelago, Fortress, Pinwheel (7 maps total)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tb8pW2KAN4oFkh1aK1BfD8
2026-10-10 18:03:24 +00:00

422 lines
34 KiB
Markdown

# Deploying Netwall
Netwall is one Node process: it serves the static game from `public/` and a WebSocket endpoint at `/ws` that runs
the co-op rooms. There is no database and no build step; games live in memory (a restart ends every running game).
```
phone / laptop ──https──▶ nginx Proxy Manager (TLS, :443) ──http + WebSocket──▶ Debian 13 LXC: node :8080
◀──wss── (HOST=::, IPv4 + IPv6)
```
What was actually run while writing and re-checking this:
* The server and its test suite (Node 20.20.2 and 24.21), the real `node server/index.js` over IPv4 and IPv6 on loopback
(`curl`, two-player WebSocket sessions, `SIGTERM` exit 0), the IPv6-disabled case in a throwaway network namespace,
and `systemd-analyze verify` on the unit (syntax only).
* **A real nginx 1.26.3** (the Debian 13 package, unpacked without root and run in a private network namespace) loaded
with the server and location blocks that **nginx Proxy Manager v2.16.0** renders: its own `nginx.conf` (timeouts,
`real_ip`, headers), `proxy.conf` and the proxy-host and custom-location templates, taken from the NPM v2.16.0 source
and filled in by hand. Behind it ran the real game server, headless Chromium (page, service worker, co-op lobby, the
e2e scenarios over https), `curl` and raw WebSocket clients, with extra loopback addresses acting as distinct client
and proxy addresses (`203.0.113.x`, `2001:db8::x`, `192.168.77.9`). Everything marked **observed** below comes from
that setup.
* NPM's UI labels (Details / Custom Locations / SSL tabs, "Websockets Support", "Cache Assets", "Forward Hostname / IP",
the gear icon for the Advanced box) and its path-prefix handling were **read from the v2.16.0 source**
(`frontend/src/...`, `backend/internal/nginx.js`), not clicked through. Those are marked **from source**.
**Never run:** the NPM application itself (its UI, its Docker image, its database), NPM versions other than 2.16.0,
Let's Encrypt, NPM in Docker with published ports (see "Client addresses" in section 4), a real phone, a Proxmox LXC, a
started systemd unit, a git deploy key. Labels differ between NPM versions, so check each step against yours.
## 1. Requirements
* Debian 13 (trixie) LXC, with its own IPv4 (and optionally IPv6) address on a network the proxy can reach.
* Node 18 or newer. Debian 13 ships Node 20.19 as `nodejs`. The server is plain CommonJS with one runtime
dependency (`ws`).
* A DNS name for the game (for example `game.example.com`) pointing at the proxy.
## 2. Install in the LXC
Run as root inside the container.
```sh
apt update
apt install -y nodejs npm # Debian 13: Node 20.19
node --version
# a dedicated, login-less user that only runs the game
useradd --system --home-dir /opt/netwall --shell /usr/sbin/nologin netwall
# copy the project to /opt/netwall (from your workstation, for example):
# rsync -a --exclude node_modules --exclude test --exclude .git ./ root@<lxc>:/opt/netwall/
# it needs server/, public/, package.json, package-lock.json and deploy/ (for the unit file in section 3; docs/ is optional)
# or clone it there: `git clone <repository url> /opt/netwall`. That works from inside the LXC only if the LXC can read the
# repository: a private one needs an SSH key on the LXC that the git server knows (a read-only "deploy key"); see the README.
cd /opt/netwall
npm install --omit=dev # only "ws"; `npm ci --omit=dev` also works and follows the lockfile exactly
chown -R root:root /opt/netwall # the service user only needs read access
# quick manual test as the service user (Ctrl-C to stop)
runuser -u netwall -- env PORT=8080 node server/index.js
```
When run directly the server prints `http://localhost:PORT/` and one URL per non-internal interface address
(IPv4 as `http://a.b.c.d:PORT/`, IPv6 as `http://[addr]:PORT/`, link-local `fe80::` skipped).
## 3. systemd service
```sh
install -m 0644 /opt/netwall/deploy/netwall.service /etc/systemd/system/netwall.service
systemctl daemon-reload
systemctl enable --now netwall
systemctl status netwall
journalctl -u netwall -f # live log
```
`deploy/netwall.service` runs as user `netwall`, `Restart=on-failure`, and sets the environment below. Edit the
`Environment=` lines (or use `systemctl edit netwall` and put them in an override) and `systemctl restart netwall`.
Hardening in the unit: `NoNewPrivileges`, an empty capability set (port 8080 needs none), `RestrictAddressFamilies`,
`RestrictRealtime`, `RestrictSUIDSGID`, `LockPersonality`, `SystemCallArchitectures=native`, `UMask=0077`. They need
no mount namespaces, so they should work in an LXC without extra settings; the unit has not been started in one yet,
so if `systemctl status netwall` shows a sandboxing error, comment the offending line out. A second block (`ProtectSystem=strict`, `PrivateTmp`, ...) is
left commented out: in an unprivileged Proxmox LXC it fails with `status=226/NAMESPACE` unless the container has
**Options > Features > Nesting** enabled. Turn nesting on, uncomment the block and restart if you want it.
Do not add `MemoryDenyWriteExecute=yes`; V8's JIT needs writable+executable memory.
## 4. Configuration
Environment variables, read when the server is started directly (`node server/index.js`):
| Variable | Default | Meaning |
|---|---|---|
| `PORT` | `8080` | TCP port. `0` picks a free one (the banner prints the real port). |
| `HOST` | `::` | Address to bind. `::` is **one dual-stack socket serving IPv4 and IPv6** (`ipv6Only:false`). `0.0.0.0` = IPv4 only. `127.0.0.1` = this container only. Any explicit address is used as given. |
| `TRUST_PROXY` | `auto` | `auto`: believe `X-Forwarded-For` / `X-Real-IP` / `X-Forwarded-Host` (and the stripped-port rule of the "Origin check") only when the TCP peer is loopback, RFC1918 (`10/8`, `172.16/12`, `192.168/16`), `fc00::/7` or link-local, **or is listed in `TRUSTED_PROXIES`**. `1` (also `true`, `yes`, `on`): always believe them, from any peer (then firewall the port). `0` (also `false`, `no`, `off`): never. |
| `TRUSTED_PROXIES` | empty | Comma separated IPs and CIDRs (IPv4 or IPv6) that `TRUST_PROXY=auto` believes **in addition to** the private ranges above, for example `100.64.0.0/10,2001:db8::6`. Needed when your proxy reaches the game from an address the built-in list does not call private: a global IPv6 address, Tailscale / CGNAT (`100.64.0.0/10`) or any other routable one. Ignored when `TRUST_PROXY` is `1` or `0`. A malformed entry stops the start-up with a message. See "Client addresses". |
| `ALLOWED_ORIGINS` | empty | Comma separated list of extra origins allowed to open the WebSocket, for example `https://game.example.com,https://play.example.org` (list every public name if the game has more than one). **Include the port when it is not 80/443**: `https://game.example.com:8443`. See "Origin check". |
| `MAX_CONN_PER_IP` | `40` | Open WebSockets per client IP; more get HTTP `429` on the upgrade. `0` disables the cap. |
| `GRACE_MS` | `60000` | How long a disconnected player keeps their slot (so a phone can come back after a screen lock or a network switch). |
Fixed in code (options of `createServer()`, not environment variables): at most 50 rooms, 20 snapshots/s per room,
a WebSocket ping every 25 s, client frames at most 2048 bytes, 60 messages/s per socket (burst 120), a socket that does
not `join` within 10 s is closed. When all 50 rooms exist, a room in which nobody is online (it is only being held for
its `GRACE_MS`) is deleted to make room for a new one, longest-abandoned first, so on a full server the grace period is
not guaranteed; rooms with somebody online are never evicted (the newcomer gets `too_many_rooms`).
What the server does, relevant to a proxy setup:
* **Real client IP** is used only for the log and the per-IP cap. With `TRUST_PROXY=auto` and a private/loopback peer
it comes from `X-Forwarded-For` (else `X-Real-IP`); otherwise it is the TCP peer. From `X-Forwarded-For` the server
takes the **rightmost entry that is not a private address**: proxies such as NPM *append* the address they saw to
whatever header the client sent, so the entries further left are the client's own claim and are never believed (a
chain that is private all the way gives its rightmost entry). IPv4-mapped IPv6 peers (`::ffff:a.b.c.d`) are shown
as plain IPv4. `X-Forwarded-Proto` is never used. See "Client addresses" below for what this does and does not protect
when the proxy is nginx Proxy Manager.
* **Player ids.** A player's id (`pid`, kept in the browser's session storage) is what lets a browser take its slot
back after a disconnect, so the server only ever sends it to its owner; the other player is shown under an opaque,
stable alias.
* **Keep-alive and dead peers.** Every 25 s the server sends a WebSocket protocol ping to each socket and terminates a
socket that did not answer the previous one (browsers answer automatically), so a phone that silently lost the
network is dropped after 25 to 50 s (observed through nginx: first ping at 24.8 s, socket closed at 49.8 s). A
terminated player stays in the room for `GRACE_MS`. The pings are for finding dead peers; they are **not** what keeps a
proxy's idle timer quiet: a joined game receives about 20 snapshots a second, also while paused, so nginx never sees
the connection idle. Observed: a paused two-player room behind a 20 s `proxy_read_timeout` stayed connected for
60 s (three timeouts' worth) with no reconnect, at 20.5 frames/s per browser. nginx Proxy Manager's own default is 90 s
(`proxy_connect_timeout`, `proxy_send_timeout`, `proxy_read_timeout` and `keepalive_timeout` in its v2.16.0
`nginx.conf`), so no timeout setting is needed for the game. A socket that never joins is closed by the server after
10 s, whatever the proxy does.
* **Origin check** (WebSocket upgrade only, path `/ws`). A request without `Origin` is allowed (non-browser clients).
Otherwise the origin host must equal the `Host` header, **or** the first entry of `X-Forwarded-Host` (only from a
trusted proxy), **or** (trusted proxy only) the `Host` header has **no port** and the origin has the same host name on
any port, **or** be listed in `ALLOWED_ORIGINS`. Comparison is on lower-case `host[:port]` with the scheme ignored
and ports 80/443 treated as "no port" (so listing `https://game.example.com` also admits `http://game.example.com`,
but not `https://game.example.com:8443`). Everything else gets `403`; the reason is logged.
The "no port" rule exists because nginx's `proxy_set_header Host $host` (which nginx Proxy Manager uses) **drops the
port**: published on `https://game.example.com:8443`, the browser's Origin is `https://game.example.com:8443` but the
server sees `Host: game.example.com`. Observed before the fix: 403 for both http and https on a non-default port
(`refused websocket ... Origin "https://highport.test:8332" does not match Host "highport.test"`), and the same
proxy host worked on 80/443; after it, both join. A browser talking to the server directly on its default
port 8080 (or any port other than 80/443) always sends that port in `Host`, so direct connections are still matched
exactly (observed: Origin `:9999` against Host `127.0.0.1:8331` is refused), and a different host name is still
refused (observed through the proxy). The one exception is a server you run on `PORT=80` or `443` itself: a direct
connection from a private address then also has a port-less `Host` and gets the same-host-name rule (other host names are still refused).
* **Path prefix.** The server only knows `/ws` and `/`; the browser builds the socket URL relative to the page, so a
proxy that strips a prefix works (section 7).
* `GET /healthz` (and `HEAD`) returns `200 ok`. Static files are only served from `public/`, with
`Cache-Control: no-cache` (revalidated by ETag), range requests (needed for audio on iOS Safari) and a strict
Content-Security-Policy. The server does not compress responses.
* Logs go to stdout (journald): room created/deleted, players joining/leaving, refused WebSockets. Player IP
addresses appear in the log. There is no per-request access log.
* **Shutdown.** `SIGTERM`/`SIGINT`: all sockets are closed with code 1001, the process exits 0 (within about a
second, forced after 5 s). Running games are lost; clients that reconnect are told the room no longer exists.
### Client addresses (per-IP cap, log) behind nginx Proxy Manager
`MAX_CONN_PER_IP` (default 40 open WebSockets per address) counts the client address the server works out: from
`X-Forwarded-For` / `X-Real-IP` when the TCP peer is a trusted proxy, otherwise the peer itself. All of the following
was **observed** against nginx 1.26.3 with NPM v2.16.0's configuration, with the cap set to 3 to make it visible:
* **A proxy on an address the built-in list does not call private** (a global IPv6 address, Tailscale / CGNAT
`100.64.0.0/10`; the built-in check says "not private" for `2001:db8::6` and `100.64.1.1`): with
`TRUST_PROXY=auto` the server believes no forwarded header. Three client addresses behind a proxy at `2001:db8::6`
got 3 sockets in total and `429` for the rest, and the log said `from 2001:db8::6` for everybody; with
`TRUSTED_PROXIES=2001:db8::6` each client address got its own budget (2 joined each, 6 in total) and the log named
`203.0.113.5`, `.6`, `.7`. `TRUST_PROXY=1` does the same without a list but trusts every peer, so firewall port 8080
to the proxy if you use it. Symptom of the problem: every log line says `from <the proxy>`, and a few players
exhaust the cap for everyone. The same untrusted-proxy case also ignores `X-Forwarded-Host` and the stripped-port
rule of the Origin check; the 403 line in the log then says `this connection is not from a trusted proxy`.
* **Public clients cannot fake an address.** A public client sending its own `X-Forwarded-For` and `X-Real-IP`
(`198.51.100.1`, `.2`) was still counted as its real address (`203.0.113.5`): 3 sockets joined, 2 refused.
* **Clients on a private (LAN or VPN) address can choose any address.** NPM's `nginx.conf` has
`set_real_ip_from 10.0.0.0/8; 172.16.0.0/12; 192.168.0.0/16; real_ip_header X-Real-IP;`, so when the connection to NPM
comes from such an address, nginx replaces its idea of the client address with the client's own `X-Real-IP` header
*before* it builds the forwarded headers. Eight sockets from one LAN address (`192.168.77.9`), each sending a
different `X-Real-IP`, all joined and were logged as `198.51.100.1` to `.8` although the cap was 3; five sockets from the
same LAN address without the header were capped at 3. The cap is therefore a guard against accidents and against
public abuse, **not** against someone on your own LAN: rely on the firewall and on who is on the network. Nothing else
in the game depends on the client address (it only affects the cap and the log).
* **LAN players that nginx sees as one address share one budget.** Players whose connections reach nginx from the same
address (a household behind one public IP, or a router in front of the proxy) count together, which is fine at 40
sockets (the game uses one socket per player). **Not run:** NPM in Docker with published ports and the default bridge
network, where nginx may see every LAN client as the Docker gateway address. If all your LAN players get `429`
there, raise `MAX_CONN_PER_IP`, or set it to `0` for a private game.
## 5. nginx Proxy Manager (NPM)
In NPM, **Hosts > Proxy Hosts > Add Proxy Host** (tab names and labels as in the v2.16.0 source: *Details*, *Custom
Locations*, *SSL*, and the gear icon at the right end of the tab row for the Advanced box).
**Details tab**
| Field | Value |
|---|---|
| Domain Names | `game.example.com` |
| Scheme | `http` |
| Forward Hostname / IP | the LXC's IP address (for example `192.168.1.50`) or a name NPM can resolve. Not `127.0.0.1` / `localhost` when NPM runs in its own container or on another machine: that would be NPM itself |
| Forward Port | `8080` |
| Cache Assets | **off** (the game files must revalidate; the server already sends `no-cache`) |
| Block Common Exploits | optional, can stay on |
| **Websockets Support** | **on** (required: it adds the `Upgrade` / `Connection` headers and HTTP/1.1 to the upstream). Observed with it off: the page loads, the browser's WebSocket gets HTTP `426`, and the lobby only says "Could not reach the game server." |
**Custom Locations tab**: leave empty (only needed for a path prefix, section 7).
**SSL tab**: SSL Certificate = *Request a new SSL Certificate* (Let's Encrypt), then switch on **Force SSL** and
**HTTP/2 Support** (HSTS is optional), agree to the terms, Save. Browsers fall back to HTTP/1.1 for the WebSocket;
that is normal. (The Let's Encrypt request itself was not run; it needs the name to resolve to your proxy and ports
80/443 reachable from the internet, see NPM's own documentation.)
**Advanced tab** (gear icon, *Custom Nginx Configuration*): nothing is required. In particular no
`proxy_read_timeout` / `proxy_send_timeout` is needed: NPM's default is 90 s, and a joined game never goes quiet
(see "Keep-alive and dead peers" in section 4). Save, then open `https://game.example.com/`.
### The Host header, ports other than 443, and what the Advanced tab can and cannot do
What NPM v2.16.0's `proxy.conf` sends upstream is `Host $host`, `X-Forwarded-Scheme`, `X-Forwarded-Proto`,
`X-Forwarded-For $proxy_add_x_forwarded_for` and `X-Real-IP`, plus `Upgrade` / `Connection` when Websockets Support is
on. Observed with an echo server behind the rendered config: `host` is the public name **without a port**, and there is
**no `X-Forwarded-Host` and no `X-Forwarded-Port`**. The browser's `Origin` and that `Host` match on 80/443, so
nothing needs configuring there.
* **Public port other than 80/443** (for example NPM published on `8443`, or Docker `-p 8443:443`): `$host` drops the
port, so Origin `https://game.example.com:8443` meets `Host: game.example.com`. Older versions of the server answered
`403` (observed: both `http://highport.test:8330` and `https://highport.test:8332` failed with `HTTP 403`; the same
host on 80/443 worked). The current server accepts the same host name on any port **when the connection comes from a
trusted proxy** (private address, or one listed in `TRUSTED_PROXIES`); observed: both now join, and a different host
name or a direct connection with a mismatching port is still refused. If your proxy is **not** trusted (a global IPv6 or
Tailscale address and no `TRUSTED_PROXIES`, or `TRUST_PROXY=0`), the log says `this connection is not from a trusted
proxy`: list the proxy in `TRUSTED_PROXIES`, or set `ALLOWED_ORIGINS=https://game.example.com:8443` **with the port**
(observed: an entry without the port does not match an origin with one).
* **You cannot add `X-Forwarded-Host` (or change `Host`) from the Advanced tab for the default location.** Observed:
`proxy_set_header X-Forwarded-Host $host;` and `proxy_set_header Host an.internal.name;` typed into the Advanced box
were silently ignored (the upstream still saw `Host: <public name>` and no `X-Forwarded-Host`), because NPM renders the
Advanced text at the `server {}` level and its own `location /` sets its own `proxy_set_header` lines, so nginx
inherits none of the server-level ones. The same two lines inside a *custom location's* gear box do take effect
(observed). From source: an Advanced box that contains its own `location / {` replaces NPM's default location
completely, and then you must repeat every proxy header and the `Upgrade` / `Connection` pair yourself.
So for an NPM user the one fix that always works is `ALLOWED_ORIGINS=https://game.example.com` (with the port if it
is not 443). `X-Forwarded-Host` is for hand-written nginx: observed with a hand-written `location /` that sets
`Host an.internal.name` and `X-Forwarded-Host $host`, `TRUST_PROXY=auto` joined, `TRUST_PROXY=0` got `403`, and
`TRUST_PROXY=0` with `ALLOWED_ORIGINS=http://rw2.test` joined again.
Optional, in the Advanced box, to compress text for phones on mobile data (the Node server does not; NPM's own
`gzip on` only covers `text/html` by default, observed):
```nginx
gzip on;
gzip_types text/css text/javascript application/javascript application/json application/manifest+json image/svg+xml;
```
Observed with this list: `/`, `js/*.js`, `css/style.css`, `sw.js`, `icons/icon.svg`, `assets/music/tracks.json` and
`manifest.webmanifest` come back `Content-Encoding: gzip`; the PNG icons and mp3 files are left alone.
Optional, if **co-op never connects but the page loads** and the browser's developer tools show the `/ws` request
answered `400 Request Header Or Cookie Too Large`: another application on the same parent domain has set big cookies,
and the browser sends them to the game too. nginx's default `large_client_header_buffers 4 8k` refuses a single header
line above 8 KB (NPM leaves that default). Observed with an 8300-byte cookie: `400` on HTTP/1.1 requests and on the
WebSocket upgrade, while the page itself still loaded over HTTP/2 (`200`); up to about 7.8 KB everything worked. Put
```nginx
large_client_header_buffers 4 16k;
```
in the Advanced box (observed: cookies up to 16000 bytes then pass, upgrade included). Do not expect more: the Node
server itself refuses request headers above about 16 KB with `431` (observed at 16.3 KB), so beyond that clear the
cookies or serve the game from a name outside the cookie's domain. The game sets no cookies of its own.
## 6. Dual stack (IPv4 + IPv6)
* **The app side is automatic.** `HOST=::` (the default) is a single dual-stack socket: IPv4 clients arrive as
`::ffff:a.b.c.d` and are logged as plain IPv4. Check both families from inside the LXC:
```sh
curl -i http://127.0.0.1:8080/healthz
curl -gi 'http://[::1]:8080/healthz'
```
* **Public side.** Publish an `AAAA` record for `game.example.com` pointing at the proxy's IPv6 address, next to the
`A` record, **after** ports 80 and 443 really answer over IPv6 (router/firewall rules, NPM's container having an
IPv6 address). A published but broken `AAAA` breaks IPv6 visitors and certificate issuance. The proxy-to-LXC hop
can stay IPv4; that is independent of what visitors use.
* **Firewall.** Only the proxy should reach port 8080 (everybody else goes through TLS). Inside the LXC, for example
with nftables (`apt install nftables`; keep your SSH rule):
```text
# /etc/nftables.conf (excerpt)
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
ct state established,related accept
iif lo accept
meta l4proto { icmp, ipv6-icmp } accept
tcp dport 22 accept
ip saddr 192.168.1.10 tcp dport 8080 accept # the NPM host (IPv4)
ip6 saddr fd00::10 tcp dport 8080 accept # the NPM host (IPv6), if it reaches the LXC over IPv6
}
}
```
`systemctl enable --now nftables`. The Proxmox firewall (Datacenter / Node / CT > Firewall) can do the same.
`TRUST_PROXY=auto` is meant for exactly this setup: only private-range peers (the proxy) are believed, plus the
addresses in `TRUSTED_PROXIES` if your proxy reaches the LXC over a global IPv6, Tailscale / `100.64.0.0/10` or other
routable address. If the port is reachable by other LAN machines they could forge `X-Forwarded-For`; the effect is
limited to the per-IP cap and the log (and see "Client addresses" for what NPM lets LAN clients do on its own).
* **IPv6 disabled in the container.** If IPv6 is switched off by sysctl (`net.ipv6.conf.all.disable_ipv6=1`) the
`::` socket still binds and still serves IPv4 (tested in a throwaway Linux network namespace with
`disable_ipv6=1`, not in an LXC). Only a kernel without IPv6 at all fails to bind `::`
(`EAFNOSUPPORT`, `EADDRNOTAVAIL`, ...); the server then logs
`IPv6 is not available (...); falling back to 0.0.0.0 (IPv4 only)` and listens on IPv4. You can also set
`HOST=0.0.0.0`. `EADDRINUSE` (something else on the port) is an error and never triggers the fallback.
## 7. Under a path prefix (`https://example.com/netwall/`)
The client never uses absolute URLs: it opens `new URL('ws', location.href)`, so at `https://example.com/netwall/`
it connects to `wss://example.com/netwall/ws`. The proxy must **strip the prefix** so the server sees `/` and `/ws`,
and the page must be opened **with the trailing slash** (`/netwall/`, not `/netwall`, or the relative URLs resolve to
the wrong place; nginx itself redirects `/netwall` to `/netwall/` for the locations below, observed). The simpler route
is a dedicated sub-domain (`game.example.com`), which needs none of this. Origin check: the browser's `Origin` is
`https://example.com` and `Host` is `example.com`, so it passes unchanged.
**Way 1, in the NPM UI: the Custom Locations tab of the proxy host.** Add a location with
| Field | Value | Why |
|---|---|---|
| Location | `/netwall/` | **with the trailing slash** |
| Scheme / Forward Port | `http` / `8080` | |
| Forward Hostname / IP | `192.168.1.50/` | **with a trailing slash after the address**: NPM turns everything after the first `/` into the path it puts on `proxy_pass`, so this renders `proxy_pass http://192.168.1.50:8080/;`, and that trailing `/` is what strips the `/netwall/` prefix |
and keep **Websockets Support** on in the Details tab (NPM copies it into each location). The Details tab still needs a
forward target for the host's own `/`; point it at whatever else the name serves.
What was run and what was not: NPM's `_location.conf` template, filled in the way its backend does it for exactly these
values, was run in nginx (observed: `/netwall/healthz` `200`, `/netwall` answered `301` to `/netwall/`, a WebSocket
join through `wss://.../netwall/ws`, and the repo's e2e smoke scenario and two-browser co-op scenario, invite link and
reconnect included, over https at `https://.../netwall/`). That a slash typed into the
Forward Hostname field is split into the forward path is **from source** (`backend/internal/nginx.js`
`renderLocations` in v2.16.0, and the field's own placeholder there, `eg: 10.0.0.1/path/`); the UI itself was not clicked.
Two near misses, both observed, that make this fail:
* Location `/netwall` **without** the trailing slash and a forward target ending in `/`: the page and its files load,
but `/netwall/healthz` and `/netwall/ws` are `404` (the upstream sees `//ws`), so co-op reports "Could not reach the
game server" (`WebSocket ... Unexpected response code: 404`).
* Forward Hostname `192.168.1.50` **without** the slash: nothing is stripped, so every `/netwall/...` request is `404`.
**Way 2: the host's Advanced tab** (gear icon). Paste this block; it adds its own location next to NPM's default one,
so the host's Details tab still needs *some* forward target. Observed working verbatim (the proxy address replaced by a
local one): `/netwall/healthz` `200`, `/netwall` `301` to `/netwall/`, `manifest.webmanifest` `200`, a music file with a
`Range` header `206`, a WebSocket join through `wss://.../netwall/ws`, and the e2e smoke and co-op scenarios over https.
```nginx
location /netwall/ {
proxy_pass http://192.168.1.50:8080/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
```
(An earlier version of this guide also had `location = /netwall { return 301 /netwall/; }` in front: redundant, because
nginx already answers `/netwall` with `301` to `/netwall/` for a prefix location ending in a slash; observed.) Check with
`curl -i https://example.com/netwall/healthz` (expect `200 ok`).
## 8. Verify
```sh
# from anywhere
curl -i https://game.example.com/healthz # 200 ok
curl -I https://game.example.com/ # 200, Content-Security-Policy etc.
curl -6 -I https://game.example.com/ # over IPv6 (once the AAAA record exists)
# WebSocket through the proxy (run in /opt/netwall, uses the installed "ws"). It creates a room that disappears
# again after the 60 s grace period. For a public name on a non-default port use that port in both places.
node -e "
const WebSocket=require('ws');
const ws=new WebSocket('wss://game.example.com/ws',{origin:'https://game.example.com'});
ws.on('open',()=>ws.send(JSON.stringify({t:'join',pid:'deploy-check-0001',name:'check'})));
ws.on('message',d=>{const m=JSON.parse(d);if(m.t==='joined'){console.log('joined',m.room);process.exit(0)}});
ws.on('unexpected-response',(q,r)=>{console.log('HTTP',r.statusCode);process.exit(1)});
ws.on('error',e=>{console.log(e.message);process.exit(1)});
setTimeout(()=>{console.log('no reply');process.exit(1)},5000);
"
# prints: joined XXXX (or HTTP 403 / 426 / 429 / 502, see section 9)
```
The snippet was run (with the host replaced by a test name and certificate checking off for the self-signed
certificate) through nginx with NPM's configuration and printed `joined XXXX`.
Then open the URL on two phones, create a room on one and enter the 4-letter code on the other.
## 9. Troubleshooting
| Symptom | Cause and fix |
|---|---|
| Page loads, but the game says it cannot reach the server / WebSocket never opens | **Websockets Support** is off in the Proxy Host, or something in front strips the `Upgrade` header. Turn it on. |
| The lobby only says "Could not reach the game server." | That one message covers every failed WebSocket upgrade. The browser's developer-tools console shows the real reason as `Unexpected response code: NNN`; the rows below are by that code. The server log (`journalctl -u netwall -f`) names the reason for 403 and 429. |
| WebSocket upgrade answered with **426** | **Websockets Support** is off in the Proxy Host (observed). Turn it on. |
| WebSocket upgrade answered with **403** | Origin check: the browser's `Origin` host differs from the `Host` the server sees. The log says `refused websocket from ...: Origin "..." does not match Host "..."`. On a **non-default public port** this is handled automatically for a trusted proxy (section 4, "Origin check"); if the log says `this connection is not from a trusted proxy`, list the proxy in `TRUSTED_PROXIES` (or set `TRUST_PROXY=1`). Otherwise (a proxy that rewrites `Host`, several public names) set `ALLOWED_ORIGINS=https://game.example.com` (comma separated for several, **with `:port` when it is not 80/443**) and restart. |
| Upgrade answered with **429** | More than `MAX_CONN_PER_IP` sockets from one address. If the log shows every visitor as the proxy (`from <the proxy's address>`), the proxy is not trusted: it connects from an address the `auto` list does not call private (global IPv6, `100.64.0.0/10`, ...). Set `TRUSTED_PROXIES=<its address>` (section 4, "Client addresses"). If all LAN players share one address in front of nginx, raise `MAX_CONN_PER_IP`. |
| Page loads, upgrade answered with **400** `Request Header Or Cookie Too Large` | Big cookies from another application on the same parent domain. Add `large_client_header_buffers 4 16k;` in the Advanced box (section 5), or clear the cookies. |
| WebSocket dies after exactly 60 s (or another round number) | Should not happen with the defaults: a joined game receives about 20 snapshots a second, so a proxy idle timer never fires (observed with a 20 s `proxy_read_timeout`). If it does, something else sits in the path that drops connections after a fixed time (a CDN, a second proxy, a firewall, a `proxy_read_timeout` / `proxy_send_timeout` set in the Advanced box that is far lower). Check every hop. |
| Players are dropped every few seconds and reconnect | The current client treats a socket as dead after 5 s without data (snapshots arrive 20 times a second). Something is buffering or throttling the stream; look at the proxy's buffering settings and at CPU use of the LXC. |
| **502 Bad Gateway** | NPM cannot reach the upstream. (A browser that has loaded the game before still gets the cached app and can play solo while this lasts: observed in Chromium through nginx for 502; 503 and 504 take the same path in the service worker and are covered by a unit test.) `systemctl status netwall`; from the NPM host run `curl -i http://<lxc>:8080/healthz`; check forward IP/port, the firewall (section 6) and that `HOST` is not `127.0.0.1`. |
| `listen EADDRINUSE` in the log, service restarts in a loop | Another process uses the port (`ss -ltnp | grep 8080`), or two copies are running. Change `PORT` and the NPM forward port together. |
| `IPv6 is not available (...); falling back to 0.0.0.0` | The kernel has no IPv6 at all. IPv4 keeps working; nothing to do, or set `HOST=0.0.0.0` to silence it. With IPv6 merely disabled by sysctl the `::` bind still works. |
| IPv6 visitors cannot connect, IPv4 can | The `AAAA` record points somewhere that does not answer on 443, or NPM/the firewall is not listening on IPv6. Test `curl -6 -I https://game.example.com/`. The app itself is dual-stack. |
| `status=226/NAMESPACE` when starting the service | You enabled the optional namespace-based hardening in an LXC without Nesting. Enable Proxmox **Options > Features > Nesting** for the container, or comment that block out again. |
| Old version of the game after an update | NPM *Cache Assets* is on, or a browser/service layer ignores `no-cache`. Switch Cache Assets off, hard-reload. |
| Phone loses the game when it sleeps | Expected: the socket dies. The client reconnects with the same player id and gets its slot back if that happens within `GRACE_MS` (60 s by default). Raise `GRACE_MS` for longer. |
| `Cannot find module 'ws'` | `npm install --omit=dev` was not run in `/opt/netwall`. |
## 10. Updating
```sh
systemctl stop netwall
# replace server/ and public/ (rsync as in section 2), then
cd /opt/netwall && npm install --omit=dev
systemctl start netwall
```
Running games end with the restart. For a quiet moment: `journalctl -u netwall | grep -E 'created|deleted'`
shows which rooms are open.