is ivr_detection available for self host and user_away_timeout?

Thanks for reaching out, Utkarsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

user_away_timeout, what does it do?

Thanks — you're asking what

user_away_timeout

does. Good question. • What it is:

user_away_timeout

is a session option on

AgentSession

that defines how long (in seconds) the system waits during silence before marking a user as

away

. • What it does: When no audio activity (or no expected interaction) is detected for the configured number of seconds, the user’s state is changed to

away

. This can be used to trigger logic like updating presence, pausing recording, routing calls differently, or prompting the agent to re-engage. • Default value: The default is 15.0 seconds. If you want, I can: • show the

AgentSession

option snippet that sets

user_away_timeout

, or • suggest a value based on your typical call length or silence patterns. Which would help most?, with interactive elements

ivr_detection

can help us detect voicemail?

Good question — I’m responding to whether

ivr_detection

can help detect voicemail. •

ivr_detection

is designed to detect Interactive Voice Response (IVR) systems and DTMF-driven menus. It helps the agent recognize and handle automated menus, not voicemail specifically. • Voicemail detection is handled differently: ◦ Silence detection is the primary method: if there’s no expected human speech for a configured threshold, the system can assume the call hit voicemail and take action. ◦ An optional LLM-based analysis can be added for higher accuracy (typically ~70–80%) to distinguish voicemail from live answers. ◦

user_away_timeout

(default 15.0s) can be used to mark an agent/user as away after silence and can play a role in voicemail workflows by signaling prolonged silence. So,

ivr_detection

alone does not directly detect voicemail. For reliable voicemail handling, use silence-detection thresholds (and optionally LLM analysis) alongside

ivr_detection

as part of your outbound-call logic. Would you like: • a recommended silence threshold for voicemail detection, or • a code snippet showing how to combine

ivr_detection

,

user_away_timeout

, and LLM-based checks in an

AgentSession

? Sources: Agent session | LiveKit Documentation | Agents Framework | livekit/livekit_composite | DeepWiki | How to detect and handle silence during calls, with interactive elements

• a code snippet showing how to combine

ivr_detection

,

user_away_timeout

, and LLM-based checks in an

AgentSession

?

this would help

Thanks — I’m responding to your request for a code snippet that combines

ivr_detection

,

user_away_timeout

, and an LLM-based voicemail check in an

AgentSession

. Below is a concise example showing how you might wire these together. It uses silence detection (via

user_away_timeout

), enables

ivr_detection

, and runs an LLM-based voicemail classifier on recorded audio when prolonged silence is detected. Notes: • This is an illustrative snippet using the AgentSession options and a placeholder LLM-check function. Replace the LLM call with your actual model/endpoint and audio handling. •

user_away_timeout

defaults to 15.0 seconds; set it to the threshold you prefer for voicemail detection. • The snippet assumes you can access audio segments (or a recording) to send to the LLM for classification.

# Pseudocode / illustrative example

from livekit_agents import AgentSession, AgentState  # hypothetical imports
import time

# Placeholder function: send audio bytes or transcript to your LLM classifier
def is_voicemail_via_llm(audio_bytes_or_transcript) -> bool:
    """
    Return True if LLM classifies the audio as voicemail.
    Replace with your real LLM invocation and logic.
    """
    # Example: call LLM API, return True if LLM confidence > threshold
    # response = call_llm_api(audio_bytes_or_transcript)
    # return response['voicemail_probability'] > 0.7
    pass

# Callback fired when agent state changes (including away)
def on_agent_state_change(session, new_state, last_audio_chunk):
    if new_state == AgentState.AWAY:
        # last_audio_chunk should be the recent audio we can analyze
        # First, optionally check for IVR signals (DTMF) if available
        if session.options.get('ivr_detection'):
            # If you have a DTMF/IVR event indicator, check it here.
            # Example: if session.detected_dtmf: handle IVR flow
            pass

        # Use silence event to trigger voicemail check
        # Run the LLM-based voicemail classifier on recent audio (or transcript)
        if last_audio_chunk is not None:
            voicemail = is_voicemail_via_llm(last_audio_chunk)
            if voicemail:
                # Handle voicemail path: close session, leave voicemail note, etc.
                session.take_action_on_voicemail()
            else:
                # Likely a live agent or temporary silence — you may retry or wait
                session.notify_agent_to_reengage()

# Create AgentSession with ivr_detection and user_away_timeout
session = AgentSession(
    agent_id="agent_123",
    call_id="outbound_456",
    options={
        "ivr_detection": True,         # enable IVR/DTMF detection
        "user_away_timeout": 15.0,     # seconds of silence -> mark user away
        # other options...
    },
    on_state_change=on_agent_state_change,  # register callback
)

# Start the session (connect, start media, etc.)
session.start()

# Example flow: keep feeding audio / monitoring until session ends
try:
    while session.is_active():
        # Your application streams/receives audio here
        time.sleep(1)
finally:
    session.stop()

If you want, I can: • convert this into a concrete example for your language (Node, Go, or full Python with a real LLM call), • suggest a recommended silence threshold for outbound voicemail detection, or • show how to capture and buffer the "last audio chunk" (raw PCM or transcript) for the LLM. Which would be most helpful next? , with interactive elements

convert it into a concrete python exmaple

Thanks — I’ll convert the snippet into a concrete Python example that you can adapt. This example: • Enables

ivr_detection

and sets

user_away_timeout

. • Buffers recent audio while the call is active. • When the session becomes

away

(silence timeout), runs a simple LLM-based voicemail check on the last buffered audio (here shown as a placeholder HTTP call). • Shows actions for voicemail vs. live answer. Replace the placeholders (LLM call, audio capture, LiveKit/Agents SDK imports) with your real implementations.

# Concrete Python example (illustrative)
# Replace SDK imports with your actual LiveKit Agents SDK
from typing import Optional
import requests
import threading
import time

# --- Configuration / placeholders ---
AGENT_ID = "agent_123"
CALL_ID = "outbound_456"
LLM_VOICEMAIL_API = "https://your-llm-endpoint.example.com/classify"  # POST audio bytes or transcript
LLM_CONF_THRESHOLD = 0.7  # voicemail probability threshold

# --- Simple audio buffer to hold recent audio for analysis ---
class RollingAudioBuffer:
    def __init__(self, max_seconds=20):
        self.max_seconds = max_seconds
        self.chunks = []  # list of (timestamp, bytes)
        self.lock = threading.Lock()

    def push(self, audio_bytes: bytes):
        with self.lock:
            self.chunks.append((time.time(), audio_bytes))
            # trim by time
            cutoff = time.time() - self.max_seconds
            self.chunks = [(t, b) for (t, b) in self.chunks if t >= cutoff]

    def get_combined(self) -> Optional[bytes]:
        with self.lock:
            if not self.chunks:
                return None
            return b"".join(b for (_, b) in self.chunks)

# --- LLM voicemail classifier (replace with real call) ---
def is_voicemail_via_llm(audio_bytes: bytes) -> bool:
    if not audio_bytes:
        return False
    # Example: send audio bytes to your LLM endpoint and parse JSON response
    resp = requests.post(
        LLM_VOICEMAIL_API,
        files={"file": ("audio.wav", audio_bytes, "audio/wav")},
        timeout=15,
    )
    resp.raise_for_status()
    data = resp.json()
    # Expect response shape: { "voicemail_probability": 0.85 }
    prob = float(data.get("voicemail_probability", 0.0))
    return prob >= LLM_CONF_THRESHOLD

# --- Mock AgentSession and AgentState for illustration ---
class AgentState:
    ACTIVE = "active"
    AWAY = "away"
    ENDED = "ended"

class AgentSession:
    def __init__(self, agent_id, call_id, options, on_state_change):
        self.agent_id = agent_id
        self.call_id = call_id
        self.options = options
        self.on_state_change = on_state_change
        self.state = AgentState.ACTIVE
        self._active = True

    def start(self):
        # Start audio/media plumbing in real SDK
        print("Session started")

    def stop(self):
        self._active = False
        self.state = AgentState.ENDED
        print("Session stopped")

    def is_active(self):
        return self._active

    # Simulate SDK invoking state change
    def set_state(self, new_state, last_audio_chunk):
        self.state = new_state
        if callable(self.on_state_change):
            self.on_state_change(self, new_state, last_audio_chunk)

    # Example actions
    def take_action_on_voicemail(self):
        print("Voicemail detected — leaving voicemail note / ending call")

    def notify_agent_to_reengage(self):
        print("Likely live user — notify agent to re-engage")

# --- Application glue: buffer audio, detect silence -> AWAY ---
audio_buffer = RollingAudioBuffer(max_seconds=20)

def on_incoming_audio(audio_bytes: bytes):
    # This would be wired to your media input callback
    audio_buffer.push(audio_bytes)
    # In a real SDK, you may also feed audio into voice activity detection etc.

# Callback when session state changes (including AWAY)
def on_agent_state_change(session: AgentSession, new_state: str, last_audio_chunk: Optional[bytes]):
    print(f"State changed -> {new_state}")
    if new_state == AgentState.AWAY:
        # Check IVR flag first (DTMF/IVR handling)
        if session.options.get("ivr_detection"):
            # In real SDK, inspect any DTMF/IVR events here
            print("IVR detection enabled — check for DTMF events if available")

        # Use last buffered audio for LLM-based voicemail check
        audio_for_check = audio_buffer.get_combined()
        voicemail = False
        try:
            voicemail = is_voicemail_via_llm(audio_for_check) if audio_for_check else False
        except Exception as e:
            print("LLM check failed:", e)
            voicemail = False

        if voicemail:
            session.take_action_on_voicemail()
        else:
            session.notify_agent_to_reengage()

# --- Create and run session ---
session = AgentSession(
    agent_id=AGENT_ID,
    call_id=CALL_ID,
    options={
        "ivr_detection": True,
        "user_away_timeout": 15.0,  # seconds of silence -> AWAY
    },
    on_state_change=on_agent_state_change,
)

session.start()

# --- Example run loop: simulate audio input and silence timeout ---
try:
    # Simulate incoming audio for a few seconds
    for i in range(5):
        on_incoming_audio(b"\x00\x01\x02" * 1000)  # placeholder PCM bytes
        time.sleep(1)

    # Simulate silence: SDK would set state to AWAY after user_away_timeout
    # Here we directly simulate that event after waiting 16 seconds
    time.sleep(16)
    last_chunk = audio_buffer.get_combined()
    session.set_state(AgentState.AWAY, last_chunk)

    # Continue as needed...
    time.sleep(1)
finally:
    session.stop()

If you want, I can: • adapt this to use a real audio capture pipeline (WebRTC/PCM) and show how to produce WAV bytes for the LLM, • convert it to a Node or Go example, or • suggest a recommended

user_away_timeout

and LLM threshold for outbound voicemail detection. Which would you like next? , with interactive elements