Thanks — this is a clear request. I’ll walk you through a practical, reliable pattern to run two agents (IVR agent + normal agent) so the IVR handles automated menus/voicemail but exits when a human answers, allowing the normal agent to take over. Quick summary of the approach I’ll describe: • Have an IVR agent join first and listen for the initial audio. • Use simple detection rules (silence patterns, transcription, or an LLM-based classifier) to decide IVR vs human. • If IVR/voicemail is detected → IVR proceeds with IVR logic. • If human is detected → IVR cleanly leaves (

session.shutdown()

), and the normal agent takes over (either already joined or dispatched). Before detailed steps: do you already have a transcription/LLM pipeline available for analyzing the first few seconds of audio, or should I include how to set that up? If both options are possible, tell me which you prefer (quick silence-based detection vs. transcription+LLM classification). Implementation pattern (step-by-step) 1) Join order and listening window • Let the IVR agent join the room immediately when the outbound call is answered. • Have the IVR agent listen to the first 5–10 seconds of inbound audio (this window is configurable based on real call behavior). 2) Detection strategies (pick one or combine them) • Silence / pause detection: ◦ Treat a long, consistent prerecorded greeting or extended pause as likely IVR/voicemail. ◦ Example rule: if continuous non-speech or monotone audio > X seconds (e.g., 3–5s) or repeated tone-like patterns → likely IVR. • Transcription + keyword matching: ◦ Transcribe the first few seconds and look for phrases such as “please leave a message”, “press 1”, “to speak to”, or other menu prompts. • LLM-based classifier: ◦ Send the initial transcript (or short audio features) to an LLM classifier that returns IVR vs human with a confidence score. This handles edge cases better than simple keyword rules. 3) Decision actions • If IVR/voicemail detected: ◦ IVR agent continues with IVR flow (play prompts, collect DTMF if needed, record voicemail, etc.). ◦ Optionally end the call for all participants if appropriate (use room deletion only if you want to terminate the whole call). • If human detected: ◦ IVR agent should stop participating and leave the room cleanly: ▪︎ Call

session.shutdown()

on the IVR agent session so it stops processing audio and leaves. ◦ Ensure the normal agent becomes active: ▪︎ Option A (recommended): Have the normal agent either already joined but idle, then signal it (via room metadata or an internal flag) to start handling once IVR signals human detected. ▪︎ Option B: Dispatch/instantiate the normal agent after the IVR agent sets a room attribute / external flag indicating human answered. 4) Coordinating agents (signaling / metadata) • Use room attributes, job metadata, or an external store to record the detection result: ◦ Example: set room attribute

answer_type = "human"

or

answer_type = "ivr"

. • The normal agent watches for that attribute change (or is triggered by an event) and starts processing when it reads

answer_type = "human"

. • This avoids race conditions and provides a clear handoff. 5) Clean exit vs end-call • For IVR leaving only: call

session.shutdown()

for the IVR agent. • To end the entire call (if IVR leaves and you want to terminate for everyone): call the API to delete the room. • Don’t use

delete_room

when you intend the human and normal agent to continue the call. Practical code hints (conceptual) • IVR agent: ◦ Join room ◦ Listen first N seconds ◦ Run detection (silence/transcript/LLM) ◦ If human: ▪︎ set room metadata

answer_type = "human"

▪︎ await any final steps, then

session.shutdown()

◦ If IVR: ▪︎ set

answer_type = "ivr"

▪︎ continue IVR prompts and flows • Normal agent: ◦ Either: ▪︎ Already joined and waiting for

answer_type == "human"

before enabling voice handling ▪︎ Or subscribe to external event and join only after

answer_type == "human"

Operational tips • Tune your listening window (5–10s is a typical starting point). • Combine detection signals for better accuracy (silence + transcription + LLM confidence). • Add a short cooldown/delay so the normal agent doesn’t start speaking while IVR is still finishing a prompt. • Log detection decisions and confidence scores to iterate on thresholds. If you’d like, I can: • Provide a focused code example for your stack (Python or Node) showing how to detect, set room attributes, and call

session.shutdown()

. • Draft a sample detection classifier prompt or rules for the LLM/transcription-based approach. Which would help you most next? Sources: Agents telephony integration | LiveKit Documentation | How to detect and handle silence during calls | LiveKit