Add resource limits, abuse protection and overload handling
Running games always continue; new work is refused when there is no
capacity; abuse is rejected immediately.
- Docker: CPU/memory/pids caps per container (overridable in .env);
uvicorn message size, queue and concurrency limits
- Server abuse limits: connections total/per IP (close 1013), message
rate per connection (token bucket, close 1008 on floods), lobby
creation rate per IP
- Load monitor: event loop lag, CPU of the own cgroup quota and
throttling, with hysteresis (normal/busy/overloaded, severe lag
overloads immediately)
- Admission control: create_lobby and start_game refused while
overloaded or MAX_ACTIVE_GAMES are running; server_status message
broadcast on changes; /health reports load but stays ok
- AI bridge: timeout sent with requests, in-flight cap, circuit breaker
- AI service: bounded workers and queue ("overloaded"), drops expired
requests, degrades to the heuristic agent when busy or too slow
- Website: busy notices, Create/Start disabled when not accepting
games, slow retry when the server is full
- Tests for limiters, load levels, admission control and AI service
limits; docs updated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -9,3 +9,25 @@ ALLOWED_ORIGINS=
|
||||
AI_AGENT=model
|
||||
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
# --- Resource limits (hard caps per container) -------------------------------
|
||||
SERVER_CPUS=1.0
|
||||
SERVER_MEMORY=512M
|
||||
AI_CPUS=1.0
|
||||
AI_MEMORY=1G
|
||||
WEBSITE_CPUS=0.25
|
||||
WEBSITE_MEMORY=64M
|
||||
|
||||
# --- Capacity -----------------------------------------------------------------
|
||||
# Games running at the same time; more game starts are refused ("server busy").
|
||||
MAX_ACTIVE_GAMES=50
|
||||
# Open websocket connections in total / per client IP.
|
||||
MAX_CONNECTIONS=500
|
||||
MAX_CONNECTIONS_PER_IP=10
|
||||
|
||||
# --- AI service ---------------------------------------------------------------
|
||||
# Decisions computed in parallel, waiting requests before refusing, and queue
|
||||
# depth from which the cheap heuristic agent is used instead of the model.
|
||||
AI_WORKERS=1
|
||||
AI_MAX_QUEUE=64
|
||||
AI_DEGRADE_QUEUE=8
|
||||
|
||||
Reference in New Issue
Block a user