Add resource limits, abuse protection and overload handling
Running games always continue; new work is refused when there is no
capacity; abuse is rejected immediately.
- Docker: CPU/memory/pids caps per container (overridable in .env);
uvicorn message size, queue and concurrency limits
- Server abuse limits: connections total/per IP (close 1013), message
rate per connection (token bucket, close 1008 on floods), lobby
creation rate per IP
- Load monitor: event loop lag, CPU of the own cgroup quota and
throttling, with hysteresis (normal/busy/overloaded, severe lag
overloads immediately)
- Admission control: create_lobby and start_game refused while
overloaded or MAX_ACTIVE_GAMES are running; server_status message
broadcast on changes; /health reports load but stays ok
- AI bridge: timeout sent with requests, in-flight cap, circuit breaker
- AI service: bounded workers and queue ("overloaded"), drops expired
requests, degrades to the heuristic agent when busy or too slow
- Website: busy notices, Create/Start disabled when not accepting
games, slow retry when the server is full
- Tests for limiters, load levels, admission control and AI service
limits; docs updated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -5,7 +5,8 @@ messages are plain dicts:
|
||||
|
||||
- ``{"type": "welcome", "player_id", "token", "name"}``
|
||||
- ``{"type": "state", "lobby": LobbyView | null, "game": GameView | null, "events": [...]}``
|
||||
- ``{"type": "error", "message"}``
|
||||
- ``{"type": "error", "message", "code"?}``
|
||||
- ``{"type": "server_status", "load", "accepting_games"}``
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -75,5 +76,16 @@ ClientMessage = Annotated[
|
||||
client_message_adapter: TypeAdapter[ClientMessage] = TypeAdapter(ClientMessage)
|
||||
|
||||
|
||||
def error(message: str) -> dict:
|
||||
return {"type": "error", "message": message}
|
||||
# Machine readable error codes (the frontend reacts to them).
|
||||
ERROR_BUSY = "busy" # server has no capacity for this right now
|
||||
ERROR_RATE_LIMITED = "rate_limited" # client sends too many messages / creates too many lobbies
|
||||
|
||||
# Websocket close code "Try Again Later" (RFC 6455 registry), used when full.
|
||||
CLOSE_TRY_AGAIN_LATER = 1013
|
||||
|
||||
|
||||
def error(message: str, code: str | None = None) -> dict:
|
||||
message_dict: dict = {"type": "error", "message": message}
|
||||
if code:
|
||||
message_dict["code"] = code
|
||||
return message_dict
|
||||
|
||||
Reference in New Issue
Block a user