Architecture
Two planes — a worker fleet and a dispatcher — with a transport-oblivious session.
agents-go separates what runs a conversation from what decides where it runs. A session drives one conversation and knows nothing about the network; a worker runs sessions as jobs; a dispatcher places jobs onto the fleet. That split is what lets one agent scale from a laptop to a horizontally-scaled fleet without rewriting it.
Why it matters
Voice agents are long-lived, stateful, and bursty. You need to add capacity, deploy new versions, and lose a machine — all without dropping live calls. Baking the transport or the placement logic into your agent makes every one of those operations a rewrite. agents-go keeps the agent code identical whether it's a console pipe in a test or one of hundreds of workers behind a dispatcher.
The two planes
- Control plane — dispatcher + worker. A worker registers with a dispatch source, reports load, answers availability, and runs assigned jobs (goroutine-per-job). The dispatcher does load-aware selection and session pinning.
- Data plane — session + transport. An
AgentSessionbinds to a transport's audio/text I/O and drives the turn loop. The session is transport-oblivious: the same session runs over a raw WebSocket, a LiveKit/WebRTC room, a Twilio media stream, or a console PCM pipe. Swapping transports never touches agent code.
The single-binary MVP
The MVP is one binary with the dispatcher embedded in-process — no second service, no nginx.
Two seams are designed so nothing about the agent changes when you scale:
- worker ↔ dispatch is message-passing, no shared mutable state. Even in-process, the worker and dispatcher only exchange messages. Extracting the dispatcher into its own service (worker↔dispatch over the wire) is a transport swap, not a code change.
- the session is transport-oblivious (above).
Keep the single binary until scale forces the issue; extraction to a standalone dispatch service happens only when one process can't hold the fleet, and the message-passing seam makes that migration mechanical.
LiveKit-optional
The worker speaks a LiveKit-compatible protocol, so one binary can run behind its own embedded dispatcher or behind a LiveKit server — you choose the dispatch source at startup. LiveKit is an option, not a dependency: nothing in the core imports it.
Where the heavy stuff lives
The core stays small and pure. Provider SDKs, WebRTC, and telephony live in
plugin subpackages (plugins/openai, plugins/livekit/roomio,
plugins/twilio, …), so importing the core never pulls a media stack.
No ML in the Go binary. VAD and silence detection run in Go, but heavier end-of-turn / interruption ML models run in an external sidecar over HTTP — the agent process stays free of ONNX/CGo.
Related
- Speech & the turn — inside one session.
- Deploy a worker · Scaling & load
- Reference: Worker & dispatch.