Files
Wizard/docs/08-operations.md
marvinandClaude Opus 5.5 9ffee8b952 Document limits, load control and performance
- New docs/09-limits-and-performance.md: principles, all protection
  layers, what players see, configuration reference, measurements,
  tuning guide, monitoring and how to test
- Server chapter summarises the limits and links to chapter 9; circuit
  breaker description and AI_MAX_INFLIGHT default corrected
- Architecture: new files in the layout, design decisions for load
  handling; docs index lists chapter 9
- Protocol, AI and operations chapters link to chapter 9; test counts
- Root and component READMEs describe the limits and settings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:04:21 +02:00

11 KiB
Raw Permalink Blame History

8. Deployment, development & operations

Docker

Images

Service Dockerfile Build context Listens on Notes
wizard_website wizard_website/Dockerfile wizard_website/ 8082 (nginx) multi-stage: node build → nginx; build arg VITE_WS_URL (optional)
wizard_server wizard_server/Dockerfile wizard_server/ 8083 (uvicorn) installs requirements.txt, then game_logic/; runs as non-root; health check on /health
wizard_ai wizard_ai/Dockerfile wizard_ai/ plus named context game_logic – installs the game engine from the extra context; runs python -m wizard_ai serve

The AI image needs the game engine, which lives in the server folder. Compose passes it as an additional build context (requires docker compose ≥ 2.17):

build:
  context: ./wizard_ai
  additional_contexts:
    game_logic: ./wizard_server/game_logic

Building it by hand: docker build --build-context game_logic=wizard_server/game_logic wizard_ai.

Networks

Network Type Members
wizard_website external (shared with Caddy) wizard_website
wizard_api external (shared with Caddy) wizard_server
wizard_internal created by compose, internal: true (no internet) wizard_server, wizard_ai

No ports are published to the host; only Caddy reaches the containers.

Environment (.env next to compose.yaml)

Compose reads .env automatically; .env.example is only a template and is not read by anything.

Variable Used by Required Meaning
AI_SHARED_SECRET server + AI yes (compose refuses to start without it) shared secret for /internal/ai
ALLOWED_ORIGINS server no e.g. https://game.marvinkrausser.com
AI_AGENT AI no (default model) model, heuristic or random
LOG_LEVEL server + AI no INFO, DEBUG, …
SERVER_CPUS / SERVER_MEMORY compose no (1.0 / 512M) hard caps of the server container
AI_CPUS / AI_MEMORY compose no (1.0 / 1G) hard caps of the AI container
WEBSITE_CPUS / WEBSITE_MEMORY compose no (0.25 / 64M) hard caps of the website container
MAX_ACTIVE_GAMES server no (50) games running at the same time; more starts are refused ("server busy")
MAX_CONNECTIONS / MAX_CONNECTIONS_PER_IP server no (500 / 10) open websocket connections
AI_WORKERS / AI_MAX_QUEUE / AI_DEGRADE_QUEUE AI no (1 / 64 / 8) parallel decisions, waiting requests before refusing, queue depth from which the heuristic is used

How the limits work together, the full list of settings, measurements and a tuning guide are in chapter 9. Further server settings (AI_MOVE_DELAY, TRICK_PAUSE, load thresholds, …, see "Configuration" in the server chapter) can be added to the environment: section of wizard_server in compose.yaml.

AI models

./wizard_ai/models is mounted read-only to /srv/models; the service loads /srv/models/latest/ (a policy checkpoint directory). Put a new checkpoint there and restart wizard_ai — no rebuild needed. Without a checkpoint the heuristic agent is used.

Deploying

cp .env.example .env            # once; set AI_SHARED_SECRET
docker compose up -d --build    # build and (re)start everything
docker compose up -d --build wizard_website   # only one service
docker compose logs -f wizard_server
docker compose ps               # server shows "healthy" when /health answers

Reverse proxy (Caddy)

The existing Caddy container serves the game under game.marvinkrausser.com:

game.marvinkrausser.com {
    import common

    redir /wizard /wizard/website/ permanent
    redir /wizard/ /wizard/website/ permanent
    redir /wizard/website /wizard/website/ permanent

    @wizard_api path /wizard/api /wizard/api/*
    handle @wizard_api {
        uri strip_prefix /wizard/api
        reverse_proxy wizard_server:8083
    }

    handle_path /wizard/website/* {
        reverse_proxy wizard_website:8082
    }

    handle {
        respond 404
    }
}

What each part means for the app:

  • The website must be opened with a trailing slash (/wizard/website/) so relative asset paths resolve — the redirects take care of that.
  • Both prefixes are stripped, so the containers do not know about /wizard/…. The frontend finds the API through the relative URL ../api/ws.
  • Websockets are proxied automatically by Caddy.
  • /wizard/api/internal/ai is reachable through this block, but the server refuses it because the request carries Caddy's forwarding headers.

URLs: game https://game.marvinkrausser.com/wizard/website/, health check https://game.marvinkrausser.com/wizard/api/health.

Local development

One Python virtual environment for server and AI:

python -m venv .venv && . .venv/bin/activate
pip install -r wizard_server/requirements-dev.txt -r wizard_ai/requirements-dev.txt

requirements-dev.txt installs the game engine editable (-e), so changes to wizard_game are picked up immediately.

Run the three parts in three terminals:

# 1. server on :8000 (add AI_MOVE_DELAY=0 TRICK_PAUSE=0 to make AI players instant)
cd wizard_server && AI_SHARED_SECRET=dev uvicorn app.main:app --reload

# 2. AI service
cd wizard_ai && AI_SHARED_SECRET=dev SERVER_URL=ws://localhost:8000/internal/ai python -m wizard_ai serve

# 3. website on :5173 (proxies /api → localhost:8000 without the /api prefix)
cd wizard_website && npm install && npm run dev

Open http://localhost:5173. To play as several humans, use several browser tabs (each tab is its own session). Without step 2 the game still works; AI seats then play random moves.

Tests and checks

cd wizard_server && pytest      # game engine (game_logic/tests) + server (tests/)
cd wizard_ai && pytest
cd wizard_website && npm run lint && npm run build
Suite Count Covers
game engine 52 rules, full random games for 3–6 players, invariants, JSON
server 33 websocket flows incl. a complete game, reconnect, AI endpoint security, abuse limits, overload handling, load levels
AI 17 encoding, agents, env, trainer, checkpoints, service limits

Troubleshooting

Symptom Cause / fix
Browser console: Refused to apply style … MIME type (''), 404 for /assets/… The page loads assets from the wrong path. The build must use relative paths (base: './' in vite.config.ts) and the page must be opened with a trailing slash. Rebuild the website image after changing the Vite config.
Red banner "Connection lost. Reconnecting…" forever The websocket URL does not reach the server. Check https://…/wizard/api/health; check that the Caddy block strips /wizard/api; check ALLOWED_ORIGINS (must contain the exact page origin, e.g. https://game.marvinkrausser.com).
Banner "This game was opened in another window" The same session (token) was opened in a second tab; reload this tab to take it back.
Lobby shows "The AI service is offline" wizard_ai is not connected. docker compose logs wizard_ai: Server rejected the connection, check AI_SHARED_SECRET → secrets differ; connection refused → server not running or wrong SERVER_URL.
AI players only make weird/random moves Same as above (random fallback), or the AI answers too slowly (AI_DECISION_TIMEOUT), see server logs "Playing random move for player …".
Game waits and nothing happens It is a disconnected human's turn (seat shows "offline"). The game waits until they reconnect or their session expires (SESSION_TTL, then an AI takes over).
All lobbies gone The server restarted — state is in memory only.
Start page / lobby says "The server is busy", Create/Start disabled The server is overloaded or MAX_ACTIVE_GAMES games are running. Check https://…/wizard/api/health (load, cpu, throttled, active_games) and docker stats. Running games are not affected. If it happens often, raise SERVER_CPUS or MAX_ACTIVE_GAMES.
Banner "The server is full right now" Connection limit reached (MAX_CONNECTIONS, or MAX_CONNECTIONS_PER_IP — e.g. many tabs behind one NAT). The page retries on its own every 15–60 s.
Error toast "Too many messages, slow down." A client exceeded the message rate limit (10/s). Normal play never gets near it; a script or broken client might.
AI plays noticeably worse under load The AI service switched to the heuristic because it was busy (AI_DEGRADE_QUEUE) or the model was too slow; see docker compose logs wizard_ai ("Busy … using heuristic agent"). Raise AI_CPUS/AI_WORKERS or use a smaller model.
Tests pass but behaviour does not change after editing wizard_game A non-editable copy of wizard_game is installed in your virtualenv and shadows the source. pip uninstall wizard-game && pip install -e wizard_server/game_logic. Check with python -c "import wizard_game; print(wizard_game.__file__)".
Server tests hang A test opened a websocket without a context manager; use client.sockets.enter_context(client.websocket_connect(...)).

Recipes

Add a new game option (e.g. a rule variant)

  1. wizard_game/state.py: add a field to GameOptions and to to_dict / from_dict.
  2. wizard_game/game.py: implement the behaviour using self.options.<field>; add tests in game_logic/tests/test_game.py.
  3. wizard_server/app/protocol.py: add the field to SetOptions; service.py: pass it into GameOptions(...).
  4. wizard_website/src/api/types.ts: add it to GameOptions and the set_options message; LobbyScreen.tsx: add a checkbox (copy the "Hidden bids" one).
  5. wizard_ai/wizard_ai/model/encoding.py: add a feature (and update FEATURE_SIZE) if the AI should know about it; __main__.py: add a --flag to train.

Add a new client message

  1. protocol.py: new pydantic model, add it to the ClientMessage union.
  2. service.py: handle it in _dispatch (usually by calling a new Lobby method and then lobby.broadcast()).
  3. types.ts: add it to ClientMessage; useWizard.ts: add a function to actions.
  4. Add a test in wizard_server/tests/test_server.py.

Check resource usage

docker stats wizard-wizard_server-1 wizard-wizard_ai-1 wizard-wizard_website-1
curl -s https://game.marvinkrausser.com/wizard/api/health

To test the overload behaviour without real load, start the server with LOAD_FORCE_LEVEL=overloaded (new games are refused, the UI shows the busy notice).

Change how fast AI players move

Set AI_MOVE_DELAY (per move) and TRICK_PAUSE (after a trick) for wizard_server in compose.yaml.

Add another AI agent

Subclass Agent in wizard_ai/wizard_ai/agents/, add it to AGENT_NAMES and make_agent in agents/__init__.py, then use it via AI_AGENT=<name> or in evaluate --agents.

Change container ports

Website: listen in wizard_website/nginx.conf and EXPOSE in its Dockerfile. Server: --port, EXPOSE and the health check URL in wizard_server/Dockerfile, and SERVER_URL of wizard_ai in compose.yaml. Then update the Caddy block.