Hey, if we enable ivr_detection=True should that a...
# ask-ai
n
Hey, if we enable ivr_detection=True should that automatically register all tools for handling the IVR/ call trees in out agent (https://github.com/livekit/agents/blob/859096e972dbaa1d5cf481ba9522f995e0e6180c/livekit-agents/livekit/agents/voice/agent_session.py#L249) ? We see some documentation that makes it sound like it should and some documentation that reads like you should register tools for this.
t
Thanks for reaching out, James. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
n
cc @acoustic-lunch-30313
t
Thanks — good question, and I can see why the docs felt a bit contradictory. Short answer: Enabling
ivr_detection=True
does not automatically register all IVR/call-tree tools for your agent. It enables detection/heuristics for IVR environments, but you still need to register and implement the actual interaction tools (for example, DTMF senders/handlers) so the agent can navigate menus. What this means for you • If you only set
ivr_detection=True
, the agent can detect that an IVR is present, but it won’t automatically be able to press keys or run menu navigation flows. • To interact with an IVR you must register tools such as DTMF-related tasks or your own `send_dtmf_code`/`GetDtmfTask` implementations and wire them into the agent session. If you'd like, I can: • Review the specific agent code you're using and point to where to register the DTMF/menu tools. • Provide a small example of registering a DTMF tool and calling it from the agent session. • Help map the IVR menu steps you need into tool calls if you share the call-tree or expected DTMF sequences. Which would be most helpful next? Sources: Agent session | LiveKit Documentation | Tasks and task groups | LiveKit Documentation
Thanks — I see you’ve CC’d Nick. I’m responding to your original question about whether setting
ivr_detection=True
will automatically register all IVR/call-tree tools for the agent. Short answer: ivr_detection=True does not automatically register the tools needed to interact with IVRs. It enables IVR detection (so the agent can notice an IVR), but you still need to register and implement the actual tools (for example, DTMF senders/handlers like
send_dtmf_code
or
GetDtmfTask
) that perform menu navigation and tone sending. Would you like help with one of the following next steps? • I can show the minimal set of tools you should register for DTMF-based IVR navigation and an example of how to register them in your agent. • I can point out the exact spots in your agent code where to plug those tools (if you share which agent implementation/version you’re using). • If you prefer, I can draft a small example flow that detects an IVR and then uses registered DTMF tools to navigate a menu. Which option would be most helpful for you and Nick?
a
Could you give us a snippet function of how this would work in practice?
t
Thanks — I’ll show a concise, practical example of how to enable IVR detection and register a DTMF tool so your agent can both detect IVRs and actually interact with them (send/receive tones and navigate menus). Notes: •
ivr_detection=True
enables IVR detection only; it does not auto-register DTMF tools. • The snippet below demonstrates registering a DTMF handler tool and wiring a listener for incoming DTMF events. Adapt the integration points to match your agent class and runtime. Example (Python-style pseudocode you can adapt to your agent):
Copy code
from livekit.agents import Agent, function_tool, RunContext
from livekit import rtc

class IVRAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are an IVR assistant. Use DTMF detection to respond to user input."
        )
        # Ensure session is configured to detect IVRs when started
        self.session_options = {"ivr_detection": True}

    @function_tool()
    async def send_dtmf_code(self, context: RunContext, digit: str) -> str:
        """Tool to send a single DTMF digit from the agent (outbound)."""
        # Example: publish DTMF via room/local participant API
        # Replace with your actual RTC publish call
        await context.session.room.local_participant.publish_dtmf(code=digit)
        <http://context.session.info|context.session.info>(f"Sent DTMF: {digit}")
        return f"sent:{digit}"

    @function_tool()
    async def handle_dtmf_input(self, context: RunContext, digit: str, duration_ms: int) -> str:
        """Tool to handle an incoming DTMF digit detected in the call (inbound)."""
        <http://context.session.info|context.session.info>(f"Received DTMF '{digit}' duration {duration_ms}ms")

        # Map digits to actions / replies
        responses = {
            "1": "You selected sales. Transferring now.",
            "2": "You selected support. How can I help?",
            "3": "You selected billing. Please hold.",
            "*": "Returning to main menu."
        }
        reply = responses.get(digit, "Invalid option. Please try again.")
        # Generate a spoken reply using the agent/session TTS facility
        await context.session.generate_reply(instructions=reply)
        return reply

# Hook DTMF events from the room to call the tool
async def on_room_started(session):
    room = session.room

    @room.on("sip_dtmf_received")  # event name may vary; adapt to your SDK binding
    def _on_dtmf(event: rtc.SipDTMF):
        digit = event.digit
        duration = getattr(event, "duration_ms", 0)
        # Call the registered tool to process inbound DTMF
        # How you invoke a tool from an event depends on your agent runtime.
        session.invoke_tool("handle_dtmf_input", digit=digit, duration_ms=duration)

    # Optionally, to send a digit from code:
    # await room.local_participant.publish_dtmf(code="1")
Next steps I can help with: • Provide a concrete snippet that exactly matches your agent implementation/version (share which file/class you’re using). • Show how to register these tools in the agent constructor or bootstrap flow you use. • Give an example flow that detects IVR prompts and then uses
send_dtmf_code
to navigate menus. Which would you like me to produce next? Sources: Company directory phone assistant | LiveKit Documentation | Agent session | LiveKit Documentation | examples/dtmf | livekit/livekit_composite | DeepWiki
a
Say I'm calling a grocery store and I want so speak to a specific department. The IVR says press 1 to speak to the bakery, press 2 to speak to the butcher. Could you provide a code snippet that will handle calling any grocery story and always connecting to the butcher department?
t
Thanks — I’ll show a concise, complete Python example that places an outbound SIP call to a grocery store, waits for the call to be answered/for the IVR to start, then sends DTMF "2" to connect to the butcher department. Important points first: • Enabling
ivr_detection=True
does not auto-register DTMF tools; you must register or call the DTMF send API yourself. • This snippet uses the LiveKit Python API to place an outbound call and the participant DTMF publish method to send the tone. Prerequisites: • A configured LiveKit outbound SIP trunk and its ID (set as environment variable
SIP_OUTBOUND_TRUNK_ID
). • LiveKit credentials in env:
LIVEKIT_URL
,
LIVEKIT_API_KEY
,
LIVEKIT_API_SECRET
. • Installed SDK:
pip install livekit-api
(or your environment’s equivalent). Example — make the call and send DTMF "2":
Copy code
import asyncio
import os
from livekit import api

async def make_call_and_select_butcher(phone_number: str, room_name: str = "grocery-call"):
    lkapi = api.LiveKitAPI(
        url=os.getenv("LIVEKIT_URL"),
        api_key=os.getenv("LIVEKIT_API_KEY"),
        api_secret=os.getenv("LIVEKIT_API_SECRET"),
    )

    outbound_trunk_id = os.getenv("SIP_OUTBOUND_TRUNK_ID")
    agent_name = "grocery-agent"

    try:
        # 1) Dispatch agent into the room (so an agent participant is present)
        await lkapi.agent_dispatch.create_dispatch(
            api.CreateAgentDispatchRequest(
                agent_name=agent_name,
                room=room_name
            )
        )
        print(f"Dispatched agent '{agent_name}' to room '{room_name}'")

        # 2) Place the outbound SIP call and wait until answered
        await lkapi.sip.create_sip_participant(
            api.CreateSIPParticipantRequest(
                room_name=room_name,
                sip_trunk_id=outbound_trunk_id,
                sip_call_to=phone_number,
                participant_identity="store-ivr",
                wait_until_answered=True
            )
        )
        print(f"Call placed to {phone_number} and answered (or waiting finished).")

        # 3) Wait briefly for IVR prompt to begin (adjust timing as needed)
        await asyncio.sleep(2.0)

        # 4) Send DTMF tone "2" from the agent/local participant to select butcher
        # Note: this call uses LiveKit API helper to publish DTMF for the room/participant.
        # Replace source_participant_identity if your participant identity differs.
        await lkapi.room.local_publish_dtmf(
            room_name=room_name,
            source_participant_identity=agent_name,  # identity of the agent/local participant
            code=2,
            digit="2"
        )
        print("Sent DTMF '2' to select butcher department.")

    except Exception as e:
        print("Error during call flow:", e)
    finally:
        await lkapi.aclose()

if __name__ == "__main__":
    phone = "+1234567890"  # grocery store phone number
    asyncio.run(make_call_and_select_butcher(phone))
Notes and customization tips: • The sleep after answer (here 2s) is a simple way to allow the IVR prompt to start. For more reliable flows, listen for speech or IVR events before sending DTMF. • If you run an agent process in the same room and want the agent to keep interacting after transfer, ensure the agent stays connected and subscribed to audio. • If your integration uses a different SDK binding or you have a custom agent class, the same steps apply: place the SIP call, wait for answer, then publish/send the DTMF tone "2". • If you want the agent to detect the IVR prompt before sending DTMF, I can provide an example that listens for a speech segment or VAD event and sends the tone when a prompt ends. Would you like a version that waits for an explicit IVR prompt (speech-end) before sending DTMF, or one that integrates into your existing agent class? Sources: Outbound Calling Script | LiveKit Documentation | Building an Automated IVR Menu Caller | LiveKit Documentation
Thanks — great question. Yes: you can detect (or infer) whether LiveKit thinks it’s interacting with an IVR vs a live person, but it’s heuristic-based rather than a guaranteed binary classification. What LiveKit provides • Enable the
ivr_detection
option on your AgentSession to turn on IVR-detection logic. • When enabled, the agent uses signals like incoming DTMF tones, repeated/structured prerecorded prompts, silence/pause patterns, and other behavioral cues to infer an IVR environment. • Detection is used to change agent behavior (e.g., wait for menus, send DTMF tones, or invoke IVR-specific tools) — it does not automatically register DTMF/navigation tools for you. How you can use it in practice • Configure your session with
ivr_detection=True
when starting the agent session. • Register tools that handle DTMF and IVR flows (e.g., a
handle_dtmf_input
tool and a
send_dtmf_code
tool). • Listen for DTMF events (e.g.,
sip_dtmf_received
) and audio events (VAD/speech segments, silence) and combine those signals with the ivr detection flag to decide whether to treat the remote endpoint as an IVR. Simple decision flow you can implement 1. If you receive DTMF from the remote side soon after answer → mark as IVR likely. 2. If audio patterns look like structured, repeated prompts (short, consistent phrases) or there are predictable pauses → increase IVR confidence. 3. If you detect natural conversational speech, interruptions, or spontaneous replies → treat as human. 4. If ivr_detection is enabled and confidence exceeds your threshold, switch to IVR mode (use DTMF tools, wait for menus). Otherwise use human-mode behaviors. Example signals to check in code • Incoming DTMF events (
sip_dtmf_received
). • Silence / VAD events and durations. • STT transcripts structure (short, repetitive menu text vs natural sentences). • Repeated prompts (same text multiple times). • Timing: IVRs often speak a menu then wait for digits. Caveats • This is heuristic — not 100% reliable. Performance depends on audio quality, STT accuracy, and IVR design. • For the most reliable flows, combine multiple signals (DTMF + VAD + transcript pattern) and add a small state machine with thresholds (e.g., require two IVR indicators before switching modes). If you’d like, I can: • Show a short code snippet that wires
ivr_detection=True
with a small state machine that listens for
sip_dtmf_received
, VAD/speech segments, and flips an
is_ivr
flag when confidence passes a threshold. • Provide the DTMF handler/send snippets to plug in once IVR is detected. Which would you prefer — the snippet for detection logic, or the DTMF send/handle example wired to detection? Sources: Agent session | LiveKit Documentation | examples/dtmf | livekit/livekit_composite | DeepWiki | Agents telephony integration | LiveKit Documentation