.

Thanks for reaching out, Michael. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Do you have best practices for understanding intent in every turn, our goal is for every call that comes in we want to be able to gather enough context enough - doesnt have to be a lot - to be able to route them to the right agent. i.e there could be maintencance, lease, emergency agent etc. But we need to do a form of evaluation to understand user intent and route them to the right agent

Thanks — this is a great, practical question. I’m responding to your request for best practices to reliably detect caller intent each turn so you can route calls (maintenance, lease, emergency, etc.) with minimal friction. Below are focused, actionable best practices you can implement right away, plus a simple flow you can adapt. Key principles (what to aim for) • Capture intent as structured data (not just free text). That makes routing deterministic and auditable. • Persist minimal context in session state so it survives handoffs and truncated histories. • Use confidence scores and lightweight clarifying questions when the model is unsure. • Route immediately once you have “enough” information — don’t over-question callers. Practical pattern to implement 1. Define a fixed taxonomy of intents ◦ Example intents: emergency, maintenance, lease_inquiry, payments, general_support. ◦ Keep category names short and mutually exclusive to minimize ambiguity. 1. Turn-level intent classification (tool / function) ◦ Have the LLM call a dedicated function like

identify_intent(intent, confidence, required_details[])

. ◦ Record:

intent

,

confidence

(0–1), and any

required_details

(like apartment number, urgency). ◦ Treat this output as authoritative for routing decisions. 1. Persist intent and context in session state ◦ Store a small SessionState object:

intent

,

intent_confidence

,

collected_data

,

required_follow_up

. ◦ Use this state for routing and for the receiving agent to continue the conversation without re-asking basics. 1. Quick decision rules based on confidence ◦ If confidence >= 0.85: route immediately to the specialist agent for that intent. ◦ If 0.5 <= confidence < 0.85: ask one targeted clarification question (see examples below). ◦ If confidence < 0.5: run a small task group that asks up to two clarifying questions, then re-classify. 1. Clarifying question strategy (keep it minimal) ◦ Always ask single-choice or binary questions to reduce cognitive load and improve NL model reliability. ◦ Examples: ◦ “Is this an emergency that needs immediate assistance?” (Yes / No) ◦ “Are you reporting something that’s broken (like heater, water) or asking about lease terms?” (Report / Lease) ◦ After one clarifying answer re-run intent classification and route if confidence is sufficient. 1. Routing / handoff ◦ Once intent is confirmed, call a

route_to_specialist

tool that returns the appropriate agent (e.g., EmergencyAgent, MaintenanceAgent, LeaseAgent). ◦ Include the stored SessionState so the specialist has context (intent, urgency, collected fields). ◦ Optionally send a short warm-transfer message: “I’m routing you to Maintenance — they’ll have your unit number and the issue description.” 1. Minimal required fields per intent ◦ Define the minimum context needed to route (not to resolve): ◦ Emergency: location/unit, immediate danger? (yes/no) ◦ Maintenance: unit, issue type, accessibility (any pets?) ◦ Lease inquiry: tenant name, unit, desired topic (renewal/termination) ◦ Collect only those fields before routing. 1. Testing & evaluation ◦ Create test cases to validate the identify_intent function on varied phrasings. ◦ Use automated tests that assert the function call contains the expected intent and a reasonable confidence threshold. ◦ Periodically sample routed calls and use human review or LLM judgement to measure routing accuracy and update prompts/config. 1. Config-driven flows for scale ◦ Externalize intent flow config per client or site: ◦ clarifying question, allowed options, mapping from option → agent ◦ This allows per-tenant tuning without code changes. Lightweight example flow (sequence) 1. Incoming call text/voice → LLM analyzes last turn and calls

identify_intent(intent, confidence, required_details)

. 2. If confidence >= 0.85 →

route_to_specialist()

immediately. 3. If 0.5 ≤ confidence < 0.85 → ask 1 clarifying question from `required_details`; then reclassify. 4. If confidence < 0.5 → run a short TaskGroup with up to 2 clarifying prompts; then route. 5. Handoff to specialist with SessionState attached and a short transfer message. Prompting tips for reliable intent classification • Use few-shot examples that map user utterances to a single intent label. • Encourage concise output: return only the label, confidence, and required fields in a structured form the tool expects. • Favor closed-choice questions for follow-ups to increase model accuracy. Monitoring & iterative improvements • Track metrics: percent routed on first turn, average clarifying questions per call, routing accuracy (human-reviewed). • Use failures to find common ambiguous phrases and either expand taxonomy or add clarifying prompts. • Tune confidence thresholds over time per intent (emergency might need higher recall; lease inquiries can tolerate lower urgency). If you’d like, I can: • Help draft the

identify_intent

function signature and example prompts for each intent. • Create a short set of clarifying questions for your top 5 intents. • Suggest test cases to validate routing accuracy across common phrasings. Which would you prefer I start with — drafting the function + prompts, or building the clarifying-question set for your specific intents? Sources: Building multi-agent architectures with LiveKit agents | LiveKit | examples/bank-ivr | livekit/livekit_composite | DeepWiki | Agents and handoffs | LiveKit Documentation | Testing and evaluation | LiveKit Documentation | Prompting guide | LiveKit Documentation

so there are two things, scenario evaluation and intents which one would make sense to evaluate turn by turn, should this be a specialized agent, that runs in the background examples of scenarios are emergency, lease, maintenance etc

Thanks — this is a great clarifying question. I’ll address the difference between scenario evaluation and intent detection, recommend what to evaluate turn-by-turn, and give a simple architecture pattern (including whether you should use a specialized/background agent). Short answer • Evaluate intent turn-by-turn. Intent is the immediate label you need to route the call (maintenance, lease, emergency...). • Evaluate scenario as a session-level or high-priority overlay. Scenarios (emergency, escalation, fraud, long-running dispute) are broader and may be detected continuously but treated differently (escalation, interrupts, or special routing rules). • Use a lightweight, specialized background agent (or tool) to run continuous checks each turn and surface both: the most recent intent and any active scenario flags. Why this split works • Intent = “what does the caller want right now?” — best evaluated each turn to decide routing and next action. • Scenario = “what larger situation or policy applies to this session?” — often derived from multiple turns, stored context, and external signals (e.g., keyword hits, high severity, prior history). Scenarios drive special handling (immediate emergency transfer, priority routing, compliance workflows). • Separating them keeps routing fast and deterministic (intent) while allowing richer, multi-turn logic (scenario) to influence or override routing when needed. Practical pattern (turn-by-turn + background checks) 1. Turn processing (lightweight, always run) ◦ On each user turn run an intent classifier tool that returns: ◦

intent

(single label),

confidence

,

required_fields

. ◦ If confidence is high, route or hand off immediately. ◦ If confidence is medium/low, ask one clarifying question and re-run classifier. 1. Background scenario agent/tool (specialized, runs each turn) ◦ Runs in parallel or immediately after intent detection. ◦ Evaluates session-level signals and rules: ◦ Keyword/phrase matches for emergencies (e.g., “fire”, “I’m hurt”). ◦ Aggregated evidence (multiple mentions of the same problem, repeated escalation language). ◦ External signals (account flags, VIP tenant, prior unresolved tickets). ◦ Sets scenario flags in session state (e.g.,

scenario: emergency

or

scenario: escalation

) and a severity/priority score. ◦ Can override or elevate routing: if

scenario == emergency

, force emergency route regardless of intent confidence. 1. Routing decision (combine both) ◦ If scenario flag is high-priority (emergency), route immediately to emergency flow. ◦ Else use the most recent

intent

(with confidence rules) to route. ◦ Attach session state (intent, scenario, required fields) in handoff. When to make the background agent a separate specialized agent • Make it separate when you need: ◦ Reusable, centralized policy logic across many conversations (e.g., emergency detection rules, escalation thresholds). ◦ The ability to update scenario rules without changing intent prompts. ◦ Parallel processing so scenario detection doesn’t slow intent routing. • Keep it lightweight: it should not re-run heavy LLM prompting every turn unless needed; prefer: ◦ Efficient keyword/regex rules for critical scenarios + occasional LLM review when rules trigger. ◦ A small LLM-based check only when rules or confidence thresholds indicate ambiguity. Implementation specifics & trade-offs • Latency: Turn-level intent must be fast — keep it minimal (single function call returning structured output). • Accuracy vs. safety: Give scenarios (like emergency) a higher priority and lower threshold for escalation to avoid missing critical events. • Explainability: Store both the classifier result and scenario flag with reasons (e.g., “scenario set because phrase match: ‘I can’t breathe’”) so agents and audit logs understand why routing occurred. • Costs: Use cheap deterministic rules for obvious emergency hits; reserve LLM cycles for ambiguous cases or clarifying dialogs. Example flow (concise) 1. User turn → run

identify_intent()

→ returns intent + confidence. 2. In parallel run `scenario_detector()`: ◦ Quick rule checks (keywords, account flags). ◦ If hit, set

session.userdata.scenario

. 1. Routing logic: ◦ If

scenario == emergency

or severity > threshold → immediate emergency handoff. ◦ Else if intent.confidence >= threshold → route to that specialist. ◦ Else ask one clarifying question and re-evaluate. Suggested immediate next steps I can help with • Draft the

identify_intent

and

scenario_detector

function signatures and example outputs. • Create the minimal keyword/rule set for emergencies and escalation triggers. • Produce one sample dialog showing turn-by-turn decisions: high-confidence route, mid-confidence clarification, and emergency override. Which would you like first — function signatures + example outputs, or a sample dialog showing all three paths?