Scaling & load
Load reporting, capacity, and how jobs are placed across the fleet.
Voice sessions are long-lived and stateful, so scaling is about placing jobs on workers that have headroom and pinning a session to the worker that owns it — not round-robin. This page covers the knobs that govern that.
Load reporting
Each worker reports an effective load. By default it's concurrency-based:
ActiveJobs / MaxJobs. When the load reaches LoadThreshold, the worker reports
full and stops being offered new jobs.
| Option | Default | Effect |
|---|---|---|
MaxJobs | GOMAXPROCS | Concurrent job ceiling; denominator of the default load. |
LoadThreshold | DefaultLoadThreshold | Effective load at which the worker reports full. |
Load | concurrency-based | Custom LoadFunc(LoadInfo) float64 for CPU/memory-aware reporting. |
StatusInterval | DefaultStatusInterval | How often load/status is reported. |
Set a custom Load when concurrency isn't a good proxy for real cost — e.g. when
sessions vary widely in CPU.
Accepting jobs
OnRequest decides whether to accept an offered job (default: accept all). Use it
to reject jobs a particular worker shouldn't take (wrong region, feature flag,
capacity you're reserving).
Placement across the fleet
The coordinator provides shared cluster state — a worker registry, a load
table, and session affinity — over a generic kv.Store (in-memory by default, or
a backend like Redis via a plugin). Placement is load-aware, and a session is
pinned to its worker so its audio always lands in the right place.
Known boundary: the coordinator shares registry/load/affinity state but does not do cross-node job routing. Ingress is sticky and dispatch is local; candidate workers are tried sequentially, bounded by the caller's context. Cross-node routing is future work.
Scaling out
- Add capacity — start more workers; they register and begin reporting load.
- Bound per-worker load — tune
MaxJobs/LoadThresholdto match the box. - Remove capacity safely — drain before terminating, so live calls finish (Graceful drain).
Related
- Plugin: Redis (rediskv) — the multi-node coordinator backend.
- Architecture · Graceful drain
- Reference: Worker & dispatch