Add resource limits, abuse protection and overload handling
Running games always continue; new work is refused when there is no
capacity; abuse is rejected immediately.
- Docker: CPU/memory/pids caps per container (overridable in .env);
uvicorn message size, queue and concurrency limits
- Server abuse limits: connections total/per IP (close 1013), message
rate per connection (token bucket, close 1008 on floods), lobby
creation rate per IP
- Load monitor: event loop lag, CPU of the own cgroup quota and
throttling, with hysteresis (normal/busy/overloaded, severe lag
overloads immediately)
- Admission control: create_lobby and start_game refused while
overloaded or MAX_ACTIVE_GAMES are running; server_status message
broadcast on changes; /health reports load but stays ok
- AI bridge: timeout sent with requests, in-flight cap, circuit breaker
- AI service: bounded workers and queue ("overloaded"), drops expired
requests, degrades to the heuristic agent when busy or too slow
- Website: busy notices, Create/Start disabled when not accepting
games, slow retry when the server is full
- Tests for limiters, load levels, admission control and AI service
limits; docs updated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+28
-8
@@ -194,8 +194,11 @@ registered with `@register_policy`, and calls its `load`.
|
||||
the container (`compose.yaml`); restart `wizard_ai`. With `AGENT=model`
|
||||
(default) it is loaded automatically.
|
||||
|
||||
Model size matters: the service answers every AI move in a worker thread and
|
||||
the server waits at most `AI_DECISION_TIMEOUT` (5 s).
|
||||
Model size matters: the service answers every AI move in a worker thread, the
|
||||
container is capped at `AI_CPUS` (1 CPU by default) and the server waits at
|
||||
most `AI_DECISION_TIMEOUT` (5 s). If the model is too slow the service
|
||||
automatically answers with the heuristic instead (see below), so measure the
|
||||
decision time before deploying.
|
||||
|
||||
## Training (`training/`)
|
||||
|
||||
@@ -244,14 +247,28 @@ count as a win for everyone tied). Seeded for fair comparisons.
|
||||
|
||||
## Production service (`service/client.py`)
|
||||
|
||||
`AIService(server_url, agent, secret).run_forever()`:
|
||||
`AIService(server_url, agent, secret, fallback=…, workers=…, max_queue=…, degrade_queue=…).run_forever()`:
|
||||
|
||||
- connects to `SERVER_URL` with `Authorization: Bearer <AI_SHARED_SECRET>`,
|
||||
sends `hello`, then answers `decide` requests (protocol in
|
||||
[chapter 5](05-protocol.md#ai-protocol));
|
||||
- each request runs `agent.act` in a worker thread (`asyncio.to_thread`) behind
|
||||
a lock, so a slow model does not block the websocket and non-thread-safe
|
||||
agents stay consistent;
|
||||
- each request runs `agent.act` in a worker thread (`asyncio.to_thread`), so a
|
||||
slow model does not block the websocket;
|
||||
- **bounded work** (`handle_request`):
|
||||
- at most `AI_WORKERS` (default 1) decisions are computed at the same time —
|
||||
agents must be thread safe if you raise it;
|
||||
- at most `AI_MAX_QUEUE` (64) requests may be in progress; more are answered
|
||||
with `error` "overloaded" immediately;
|
||||
- a request whose server `timeout` passed while it waited for a worker is
|
||||
answered with `error` "expired" without computing it;
|
||||
- **degradation**: when `AI_DEGRADE_QUEUE` (8) or more other requests are
|
||||
waiting, or the primary agent's average decision time is more than half of
|
||||
the remaining time, the cheap fallback agent (`HeuristicAgent`) decides
|
||||
instead. `serve` sets the heuristic as fallback automatically unless the
|
||||
primary agent already is the heuristic. The reply's `agent` field says which
|
||||
one decided;
|
||||
- `OMP_NUM_THREADS` / `MKL_NUM_THREADS` are set to `AI_WORKERS` in
|
||||
`compose.yaml`, so numeric libraries don't start a thread per host core;
|
||||
- errors while deciding are reported back as `error` (the server then plays a
|
||||
random move) instead of crashing;
|
||||
- reconnects forever with exponential backoff (1 s … 30 s); logs a hint if the
|
||||
@@ -271,10 +288,13 @@ Environment variables (`config.py`): `SERVER_URL`
|
||||
(default `ws://localhost:8000/internal/ai`; in Docker
|
||||
`ws://wizard_server:8083/internal/ai`), `AI_SHARED_SECRET`, `AGENT`
|
||||
(default `model`), `MODEL_PATH` (default `models/latest`, in Docker
|
||||
`/srv/models/latest`), `LOG_LEVEL`.
|
||||
`/srv/models/latest`), `LOG_LEVEL`, `AI_WORKERS` (1), `AI_MAX_QUEUE` (64),
|
||||
`AI_DEGRADE_QUEUE` (8).
|
||||
|
||||
## Tests (`wizard_ai/tests/test_ai.py`)
|
||||
|
||||
Feature length and masks for 3–6 players, action-space round trip, all agents
|
||||
play legal full games, env rewards add up to scores, trainer runs and writes a
|
||||
loadable checkpoint, `make_agent` fallback, heuristic beats random.
|
||||
loadable checkpoint, `make_agent` fallback, heuristic beats random, and the
|
||||
service limits (queue full → overloaded, expired requests, degradation to the
|
||||
heuristic when busy or when the model is too slow).
|
||||
|
||||
Reference in New Issue
Block a user