import asyncio import os import logging from doten...
# ask-ai
n
import asyncio import os import logging from dotenv import load_dotenv from livekit import api load_dotenv() logger = logging.getLogger("make-call") logger.setLevel(logging.INFO) room_name = "my-room" agent_name = "test-agent" outbound_trunk_id = os.getenv("SIP_OUTBOUND_TRUNK_ID") async def make_call(phone_number): """Create a dispatch and add a SIP participant to call the phone number""" lkapi = api.LiveKitAPI() logger.info(f"Creating dispatch for agent {agent_name} in room {room_name}") dispatch = await lkapi.agent_dispatch.create_dispatch( api.CreateAgentDispatchRequest( agent_name=agent_name, room=room_name, metadata=phone_number ) ) logger.info(f"Created dispatch: {dispatch}") if not outbound_trunk_id or not outbound_trunk_id.startswith("ST_"): logger.error("SIP_OUTBOUND_TRUNK_ID is not set or invalid") return logger.info(f"Dialing {phone_number} to room {room_name}") try: sip_participant = await lkapi.sip.create_sip_participant( api.CreateSIPParticipantRequest( room_name=room_name, sip_trunk_id=outbound_trunk_id, sip_call_to=phone_number, participant_identity="phone_user", ) ) logger.info(f"Created SIP participant: {sip_participant}") except Exception as e: logger.error(f"Error creating SIP participant: {e}") await lkapi.aclose() async def main(): phone_number = "+1231231231" await make_call(phone_number) if name == "__main__": asyncio.run(main()) when user makes a call does it joins entry point like in inbound call
✅ 1
t
Thanks for reaching out, Sajjad. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding to your question about whether an outbound call created with your script will “join the entry point” the same way an inbound call does. Short answer: yes. When you create an outbound SIP participant with
create_sip_participant()
it joins the specified LiveKit room and is treated the same as an inbound SIP participant. An agent dispatched to that room will hit its normal entry point (
@server.rtc_session
/ job lifecycle) and can wait for or process the SIP participant just like with inbound calls. Notes that may matter for your flow: • Outbound calls require you to explicitly create the room/dispatch and then dial (your code’s
create_dispatch()
then
create_sip_participant()
flow is correct). • If the agent connects before the SIP participant arrives, the agent entry point can wait for the participant (e.g.,
ctx.wait_for_participant()
). • Feature support (DTMF, STT/TTS, transfers, etc.) is the same for inbound and outbound SIP participants. If you want, tell me: • Do you see different behavior when you run this code (agent not triggered, or SIP participant not visible)? • Which part of the entry-point lifecycle do you want to confirm (agent wait-for-participant, audio routing, or event callbacks)? Sources: Agents telephony integration | LiveKit Documentation | Accepting calls overview | LiveKit Documentation | Agent dispatch | LiveKit Documentation | Job lifecycle | LiveKit Documentation | Introduction | LiveKit Documentation, with interactive elements
n
can you explain end to end the inbound and out callflow
t
Thanks — I’ll explain end-to-end inbound and outbound call flows in LiveKit, step-by-step, and highlight where they converge and differ. Inbound call flow (SIP -> LiveKit): 1. SIP INVITE arrives at LiveKit’s SIP gateway. 2. SIP gateway matches the call to a dispatch rule (based on trunk, DID, or routing). 3. LiveKit creates or selects a room for the call (a room can be auto-created for the inbound call). 4. A SIP participant is created and joined to the room representing the phone endpoint. This triggers the normal participant lifecycle events (participant joined). 5. A dispatch may be created or looked up to assign an agent to the room according to your dispatch rules. 6. An agent is dispatched and connects to the room, which triggers the agent entry point (your server-side handler such as
@server.rtc_session
). 7. The agent’s entry point may: ◦ Immediately begin handling audio (STT/LLM/TTS pipeline), or ◦ Wait for the SIP participant (e.g.,
ctx.wait_for_participant()
), or ◦ Perform other routing/IVR logic before bridging audio. 1. Media is bridged between the SIP participant and the agent(s). Standard features (DTMF, transfers, noise reduction, STT/TTS) apply. 2. Call events (join/leave, DTMF, errors) are emitted and can be observed by your app for logging, analytics, or orchestration. 3. When the call ends, participant leave events fire and the room may be cleaned up per your retention/cleanup rules. Outbound call flow (LiveKit -> SIP): 1. Your application creates or chooses a room (explicitly or via API). 2. You dispatch an agent to that room using the Agent Dispatch API (e.g.,
create_dispatch()
), which notifies and/or connects the agent; this triggers the agent entry point when the agent connects. 3. Your app calls the SIP outbound API to create a SIP participant in the room (e.g.,
create_sip_participant()
), providing the outbound trunk ID and destination number. 4. LiveKit’s SIP gateway initiates an outbound SIP INVITE to the PSTN via the configured trunk. 5. When the PSTN endpoint answers, the SIP participant joins the room and emits the same participant-joined lifecycle events as an inbound call. 6. The agent’s entry point handles the SIP participant identical to inbound: it can wait for the participant, start the voice pipeline, bridge media, or run conversational logic. 7. Media is exchanged between the agent and the remote phone via the SIP participant; DTMF, STT/TTS, transfers, etc., are supported the same way. 8. Events and telemetry are emitted as in inbound flow. Room and participant cleanup follow your policies once the call ends. Where inbound and outbound converge: • Once a SIP participant joins the room (either from an inbound INVITE or from
create_sip_participant()
answering an outbound INVITE), the participant lifecycle, media handling, agent entry point behavior, and supported features are the same. • Agent dispatch and the agent entry point run the same code paths regardless of call direction. Agents can wait for participants, process audio, and run the same pipelines. Key differences to watch for: • Triggering: inbound calls are initiated by external SIP INVITEs and routed by dispatch rules; outbound calls are initiated by your application calling the SIP outbound API. • Resource control: outbound requires your app to explicitly create dispatches/rooms and dial; inbound can auto-create rooms based on routing rules. • Timing: ensure agents are dispatched/connected at the appropriate time for your UX — agents can be connected before the SIP participant answers or after; your entry point logic should handle both cases. If you want, I can: • Diagram these flows with sequence steps and example API calls. • Show example agent entry-point logic for waiting/handling a SIP participant. • Help troubleshoot a specific timing/behavior you’re seeing when testing inbound or outbound calls. Which of these would be most helpful?, with interactive elements
n
can you make it with more clarity
t
Thanks — I’ll make the end-to-end inbound and outbound call flows clearer and more concise, with step-by-step bullets and a simple ASCII sequence for each. I’ll highlight where they match and differ. Inbound call flow (SIP -> LiveKit) 1. SIP INVITE arrives at LiveKit’s SIP gateway (from PSTN/trunk). 2. SIP gateway matches routing to a dispatch rule or DID. 3. LiveKit creates or selects a room for the call. 4. A SIP participant is created and joins the room (represents the phone endpoint). 5. Dispatch logic assigns or creates a dispatch that targets an agent and the room. 6. The agent connects to the room — this triggers your agent entry point (server-side handler). 7. Agent entry point either: ◦ Immediately processes audio, or ◦ Waits for the SIP participant (e.g.,
ctx.wait_for_participant()
), or ◦ Runs IVR/routing before bridging. 1. Media is bridged between agent and SIP participant; features like DTMF, STT/TTS, and transfers work normally. 2. Call ends → participant leave events and room cleanup per your policy. Inbound ASCII sequence: SIP INVITE -> SIP gateway -> [match rule] -> Create room -> SIP participant joins --> Dispatch -> Agent connects -> Agent entry point -> Bridge media -> End call Outbound call flow (LiveKit -> SIP) 1. Your app creates/chooses a room (explicitly). 2. Your app creates a dispatch for an agent to that room (e.g.,
create_dispatch()
). 3. Agent connects to the room — agent entry point runs. 4. Your app calls
create_sip_participant()
with trunk ID + destination number. 5. LiveKit SIP gateway sends outbound SIP INVITE to PSTN via trunk. 6. When remote answers, SIP participant joins the room (same participant lifecycle as inbound). 7. Agent entry point handles the SIP participant the same way as inbound (wait/bridge/process). 8. Media exchanged via SIP participant; same features supported. 9. Call ends → participant leave events and room cleanup. Outbound ASCII sequence: App -> Create room -> Create dispatch -> Agent connects -> App -> create_sip_participant() -> SIP INVITE outbound -> Remote answers -> SIP participant joins -> Bridge media -> End call Where they converge • Once the SIP participant is in the room, inbound and outbound behave identically: same lifecycle events, agent entry point, media handling, and features (DTMF, STT/TTS, transfers). • Agent logic and voice pipeline are reused for both directions. Key differences (short) • Trigger: inbound is started by external SIP INVITE; outbound is started by your API call. • Room/dispatch creation: inbound can auto-create; outbound requires your app to create them. • Timing: ensure agents are created/connected in the order your UX requires (agent-first or participant-first). Agent entry point should handle both timings. Next step — which do you want now? • A short agent entry-point code example that waits for the SIP participant, or • A sequence diagram with timestamps/events for a specific timing scenario you're seeing (e.g., agent connects before participant), with interactive elements
n
in case of inbound call why Waits for the SIP participant (e.g., ctx.wait_for_participant()), or Runs IVR/routing before bridging.
t
Good question — I’ll explain clearly why an agent entry point often waits for the SIP participant or runs IVR/routing before bridging in an inbound call. Why the agent may wait for the SIP participant • Timing: the agent is dispatched and its entry point can start before the caller’s SIP participant has fully joined the room. Using
ctx.wait_for_participant()
ensures the agent doesn’t act on an empty room and only starts media/session work once the caller is present. • Lifecycle events: many agent workflows rely on participant-joined events (identity, metadata) to proceed safely. Why IVR / routing often runs before bridging • Intent & triage: IVR or short voice triage gathers caller intent (menu choices, initial speech) so the system can route to the correct agent or skillset instead of blindly connecting to any available agent. • Context: routing can attach caller metadata (DID, chosen option, detected intent) to the room so the agent receives context on join. • Efficient handoff: routing decisions prevent unnecessary agent time (e.g., send billing calls to billing agents), and let the agent entry point decide whether to bridge, play prompts, or transfer. Practical sequence for inbound calls (concise) 1. SIP INVITE → SIP gateway receives call. 2. Routing/dispatch rule selected → room created or selected. 3. Agent dispatch → agent entry point starts. 4. Agent waits for SIP participant to join (or polls for join). 5. IVR/triage runs (if configured) to collect intent/data. 6. Route/decide → bridge media to the right agent(s) or take alternate action. 7. Call proceeds; events logged; cleanup on hangup. When you might skip waiting or IVR • If you always want immediate bridging and guarantees the SIP participant is already joined, you can bridge immediately. • If you want zero-latency connect and don’t need triage/context, agent logic can assume participant is present (but risk acting before join). If you’d like, I can: • Provide a short agent entry-point code example showing
wait_for_participant()
+ simple IVR handling, or • Show a sequence diagram for the exact timing you’re seeing. Which would help more? Sources: Agents telephony integration | LiveKit Documentation | Job lifecycle | LiveKit Documentation | The Handoff Pattern for Voice Agents That Replaces IVR Menus | LiveKit, with interactive elements
n
how livekit handles multi agent orchestration
t
Thanks — I’ll explain how LiveKit handles multi‑agent orchestration, clearly and concisely, with patterns, mechanics, and a short flow for each. Supervisor pattern (preferred when one agent must maintain session context) • One supervisor agent stays in control of the call and delegates focused work to short-lived Tasks. • Tasks run inside the same session and return typed results to the supervisor so conversation history and turn detection remain intact. • Use when you need a single point of awareness, safety checks, or continuous context without re-summarizing. Handoff pattern (full transfer of control) • Control is transferred completely from one agent to another (or to a human). • The original agent stops participating; context must be explicitly passed if needed. • Use for escalation, specialist handoffs, or isolated agent responsibilities to save tokens. Task-based delegation & Tooling • Define reusable Tasks (AgentTask[ResultType]) and call them from the supervisor via function-like tools. • Use TaskGroup to sequence multiple tasks with shared context. •
function_tool
is the recommended mechanism for structured agent-to-agent invocation and result passing. Dispatch, agent lifecycle, and scaling • Workers register and receive job assignments; jobs are dispatched to available agents by agent_name. • When a room is created (or SIP call arrives), LiveKit dispatches an agent job; the worker joins the room as an AGENT participant. • Multiple agents can join the same room; orchestration is managed by your agent code (supervisor, handoff, or explicit coordination). Inter-agent communication patterns • Supervisor invokes tasks or other agents via `function_tool`; this keeps responsibilities clear and allows deterministic chaining. • Use UserData to attach contextual metadata to a session (caller info, routing decisions) so agents can be stateless and horizontally scaled. Common orchestration flows (concise ASCII) Supervisor + Tasks: App/Platform -> create room & dispatch supervisor Supervisor joins -> receives caller/context Supervisor -> invoke Task A (
function_tool
) Task A returns -> Supervisor continues -> bridge media / respond Handoff: Agent A (triage) -> decide to hand off -> transfer context Agent B joins -> Agent B handles remainder (Agent A leaves) Multi-agent co-browse / parallel specialists: Agent Supervisor -> spawn Specialist A and B (Tasks or agent joins) Specialists join room -> Supervisor coordinates who speaks or merges results When to pick which pattern • Supervisor: preserve holistic context, low-latency tasking, safety, and auditability. • Handoff: clear separation of responsibilities or cost/token isolation. • Tasks: modular, testable units for specific work (NLP classification, payment handling). Best practices • Keep the supervisor as the single truth for conversation state when possible. • Use
function_tool
for structured handoffs and to avoid ad-hoc audio-level coordination. • Attach minimal context via
UserData
so agents remain stateless and scale easily. • Design entry-point code to handle agents joining before/after participants and to wait for needed participants. Would you like: • A short code example showing a supervisor invoking a Task and waiting for results, or • A diagram for your specific scenario (human + 2 AI specialists + supervisor)? Sources: How to Use the Supervisor Pattern for Multi-Agent Voice AI Systems | LiveKi... | The Handoff Pattern for Voice Agents That Replaces IVR Menus | LiveKit | Building Multi-Agent Architectures with LiveKit Agents | An open source stack for real-time multimodal AI | LiveKit, with interactive elements
n
no i am asking about the AI agen (brain)
t
Thanks — I’ll explain how the AI agent (the “brain”) works end‑to‑end and how it orchestrates multi‑agent behavior in LiveKit, clearly and concisely. Core concepts (what the brain is) • AgentSession: the runtime that manages the real‑time voice pipeline (STT → LLM → TTS), turn detection, interruptions, and session state. It is the session-level “brain manager.” • Agent: the agent’s logic and instructions (LLM prompts, tools, decision rules) that run inside an AgentSession. • Tasks: short-lived, typed units of work executed inside the same AgentSession; they perform focused actions (lookup, payment, escalate) and return structured results to the Agent. How the brain processes a call (simple flow) 1. AgentSession starts and is attached to the room. 2. Incoming audio → STT produces transcript chunks. 3. Transcript chunks → LLM processes context + tools as configured. 4. LLM output → Agent decides next action (speak via TTS, call a Task, or transfer). 5. TTS renders audio back into the room; state and conversation history persist in the session. Orchestration patterns (how multiple AI roles are coordinated) • Supervisor pattern (recommended when one central brain must keep full context): ◦ One supervisor AgentSession stays active, delegates work to Tasks (e.g., “handle_billing”), collects typed results, and continues orchestration without leaving the session. • Handoff pattern (full transfer of control): ◦ One agent exits and another agent takes over. Context must be explicitly passed to the new agent (used for escalation or isolated domains). • Observer/guardrail pattern (parallel monitoring): ◦ A background observer listens to transcripts, evaluates safety/compliance, and injects updates into the session context (does not interrupt main flow). Inter-agent communication & tools • Use function-like tools (
function_tool
) or Tasks for deterministic, structured calls between capabilities. • Tasks run inside the same session (no token-loss summarization) and return typed results the brain consumes. • Use UserData to attach minimal session metadata so agents remain horizontally scalable and mostly stateless. Practical behaviors the brain handles • Turn-taking and interruptions (preempting TTS, resuming context). • Real-time tool use (DB/API calls via
function_tool
). • Safety/compliance checks via observer agents. • Warm transfers (AI→human) via SIP/WarmTransferTask when escalation is needed. Concise example sequence (supervisor pattern) App -> create room & dispatch supervisor Supervisor (AgentSession) joins -> STT streams in Supervisor -> calls Task A (
function_tool
) -> Task A returns result Supervisor -> uses result to decide LLM response -> TTS plays to caller When to pick patterns • Supervisor: maintain continuous context, low latency, safety. • Handoff: isolated domains, token/cost containment, or when a specialist fully takes over. • Observer: add non‑blocking safety/compliance monitoring. If you want, I can next: • Show a short code example of an AgentSession that delegates a Task and handles the returned result, or • Show how to structure tools/Task signatures for predictable orchestration. Which would you like? Sources: Agent session | LiveKit Documentation | How to Use the Supervisor Pattern for Multi-Agent Voice AI Systems | LiveKi... | The Handoff Pattern for Voice Agents That Replaces IVR Menus | LiveKit | How to Build a Background Observer for Voice AI Guardrails | LiveKit | Introduction | LiveKit Documentation, with interactive elements
n
like this @function_tool() async def markspain_protocol_agent(context: RunContext_T): """ After caller have replied to the greeting, immediately call the xyz_protocol_agent to collect or confirm their information. You must only call this tool once per call """ logger.info("Transferring to xyz_protocol_agent agent") return await context.session.current_agent._transfer_to("xyz_protocol_agent", context)
t
Thanks — I’ll show a clear, practical example that matches your snippet: a
@function_tool
that transfers control from one AI agent to another exactly once per call, plus brief best-practice notes. Example: transfer tool (conceptual)
Copy code
@function_tool()
async def markspain_protocol_agent(context: RunContext_T):
    """
    After caller replies to the greeting, immediately transfer to the
    'xyz_protocol_agent' to collect/confirm info. Call this once per call.
    """
    logger.info("Transferring to xyz_protocol_agent agent")

    # Ensure we only transfer once per call
    if context.session.user_data.get("transferred_to_xyz"):
        logger.info("Already transferred to xyz_protocol_agent; skipping")
        return {"status": "skipped", "reason": "already_transferred"}

    # Mark transferred (persist minimal state)
    context.session.user_data["transferred_to_xyz"] = True

    # Perform the transfer via the current agent session API
    # (uses the agent runtime's transfer method; your SDK method may differ)
    result = await context.session.current_agent._transfer_to("xyz_protocol_agent", context)

    return {"status": "transferred", "result": result}
Key points / best practices • Single-call enforcement: persist a small flag in
context.session.user_data
(or equivalent) so the tool is idempotent and only runs once per call. • Use structured returns: return a small object describing outcome so the supervisor agent can react (retry, fallback, or continue). • Context passing:
_transfer_to(...)
should include the
context
or explicit metadata so the receiving agent gets necessary caller info. • Supervisor vs handoff: this pattern is a handoff (full transfer). If you instead want the supervisor to remain in control, invoke a Task or
function_tool
that runs the protocol logic inside the same session rather than transferring. • Observability: log the transfer and emit events so you can trace retries or failures. If you want, I can: • Convert this to your exact SDK methods (showing real names if yours differ), or • Provide the equivalent supervisor+Task example (no agent exit) so you can compare handoff vs in-session delegation. Which would you prefer?, with interactive elements
n
i am asking like langgraph ochestration framework does livekit provide any such
t
Short answer: LiveKit does not provide a visual graph/orchestration UI exactly like LangGraph out‑of‑the‑box. What LiveKit does provide that covers the same orchestration needs: • AgentSession + Agent + Tasks: a runtime “brain” that manages STT → LLM → TTS and lets an agent delegate focused work as typed Tasks inside the same session (supervisor pattern). •
function_tool
and Tasks: structured, deterministic tools for invoking logic, APIs, or other agent capabilities and returning typed results. • Handoff primitives: explicit transfer/handoff (warm transfer / agent-to-agent transfers) for full control switches. • Observer pattern: background monitors that can inject safety/compliance signals into sessions. • UserData, job dispatch, and event hooks: minimal session state, dispatch rules, and lifecycle events to coordinate agents and rooms at scale. What that means compared to LangGraph • Similarities: you can compose complex flows, call tools/tasks, chain agents, and enforce guardrails—so you can implement graph-like orchestration and deterministic pipelines using LiveKit’s APIs and Task/function_tool primitives. • Differences: there is no built-in visual orchestration/graph editor or an explicit node/edge execution engine like LangGraph. Orchestration is implemented in code (agent logic, Tasks, and transfer calls) rather than via a bundled graph UI. Options you have • Implement graph-like orchestration in code using AgentSession + Tasks +
function_tool
(supervisor pattern) for centralized orchestration. • Use LiveKit for realtime media/agent runtime and integrate an external orchestration engine (LangGraph, Temporal, or a workflow engine) to drive which agents/tasks run and when. • Build a lightweight graph layer on top of LiveKit: store a graph definition and have a supervisor agent interpret/execute nodes (call Tasks or transfer to agents) at runtime. If you want, I can: • Show a short example of implementing a graph executor inside a supervisor AgentSession that runs nodes as Tasks or
function_tool
calls, or • Outline how to integrate LangGraph (or another workflow engine) with LiveKit (events → graph triggers → agent/task invocations). Which would you like? , with interactive elements