Telephony with Twilio

The Twilio plugin turns an inbound phone call into an AgentSession bound to the call's audio. Twilio connects a Media Stream (a WebSocket carrying μ-law audio) to your server; each call becomes one session over the generic WebSocket transport.

The shape

Two endpoints:

• Voice webhook (POST) — Twilio hits this when a call comes in; you answer with TwiML telling Twilio to open a bidirectional media stream.
• Media WebSocket (GET) — each call's audio; bound to one AgentSession.

Serve the webhook

The plugin builds the TwiML and provides an HTTP handler:

import "github.com/webdeveloperben/agents-go/plugins/twilio"

http.Handle("/twilio/voice", twilio.WebhookHandler(twilio.WebhookConfig{
    // StreamURL: wss://<host>/twilio/media (derived from the request if empty)
}))

twilio.BuildTwiML(streamURL, params) is available if you build the response yourself.

Bind media to a session

Each media WebSocket becomes one AgentSession. Write the per-call logic once as a websocket.SessionFunc (func(ctx, *websocket.Transport) error): build the session bound to t.Input() / t.Output(), start it, then block on t.Wait(ctx) until the caller hangs up.

func runSession(ctx context.Context, t *websocket.Transport) error {
    session, err := agents.NewSession(agents.Config{
        LLM: "openai/gpt-4o", STT: "openai/gpt-4o-mini-transcribe", TTS: "openai/tts-1",
        Input:  agents.SessionInput{Audio: t.Input()},   // Twilio media in
        Output: agents.SessionOutput{Audio: t.Output()}, // agent audio out
    })
    if err != nil {
        return err
    }
    defer func() { _ = session.Close(ctx) }()

    agent, err := agents.NewAgent(agents.AgentOptions{Instructions: "You answer the phone."})
    if err != nil {
        return err
    }
    if err := session.Start(ctx, agents.StartOptions{Agent: agent}); err != nil {
        return err
    }

    t.Wait(ctx) // block until the caller hangs up or the socket closes
    return nil
}

Single process (start here)

websocket.Handler runs your SessionFunc directly — one process, no worker or dispatcher. A single process handles many concurrent calls.

mux.Handle("/twilio/media", websocket.Handler(twilio.New(), runSession))

The runnable examples/twilio-single-process wires both endpoints this way.

Worker fleet (horizontal scale)

When one process can no longer hold your call volume, run the same SessionFunc inside a worker and route calls through a dispatcher. websocket.NewSessionHandoff bridges the accepted socket to the worker's entrypoint, and websocket.DispatchHandler accepts the socket and dispatches a job:

handoff := websocket.NewSessionHandoff()
w, err := worker.New(worker.WorkerOptions{
    Entrypoint: handoff.Entrypoint(runSession), // the same SessionFunc
})
if err != nil {
    log.Fatal(err)
}
// ... w.Run(ctx, dispatchSource) in the background ...
mux.Handle("/twilio/media", websocket.DispatchHandler(twilio.New(), dispatcher, jobs, handoff))

The runnable examples/twiliovoice wires the worker path end to end with the in-repo fakes. See Scaling & load for when and how to grow into it.

Run it

Related

• Plugin: Twilio — webhook config and framing details.
• Transports · Architecture
• Deploy a worker