which consumes more token: a long prompt of 9 sheets or 10 function calls?

Thanks for reaching out, Utkarsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

1. Characters 19432, words 2697 2. 10 function which will let model know what to say on the basis of some parameters 3. GPT-4o realtime model 4. total tokens consumption

Thanks — this is enough to estimate. I’ll walk through the math and assumptions so you can see how total token consumption compares for the two options on GPT‑4o realtime. Quick rule of thumb • Rough conversion: about 4 characters ≈ 1 token (this is an approximation; exact token counts vary by text content). • Total tokens = input tokens + output tokens. You asked about total consumption, so I’ll include both input and a few output-size scenarios. Option A — single long prompt (19,432 characters / 2,697 words) • Estimated input tokens: 19,432 ÷ 4 ≈ 4,858 tokens. • If the model’s reply is: ◦ Short (200 tokens): total ≈ 4,858 + 200 = 5,058 tokens ◦ Medium (1,000 tokens): total ≈ 5,858 tokens ◦ Long (2,000 tokens): total ≈ 6,858 tokens Option B — 10 function calls that guide the model We need to know the size of each function call payload (the JSON/parameters sent) and the output produced per call or overall. I’ll give three example scenarios: 1) Small function payloads (e.g., 200 characters each ≈ 50 tokens) • Input per call ≈ 50 tokens → 10 calls = 500 input tokens. • If the model returns a single combined response of: ◦ Short (200 tokens): total ≈ 500 + 200 = 700 tokens ◦ Medium (1,000 tokens): total ≈ 1,500 tokens 2) Medium function payloads (e.g., 1,000 characters each ≈ 250 tokens) • Input per call ≈ 250 tokens → 10 calls = 2,500 input tokens. • With outputs: ◦ Short (200 tokens): total ≈ 2,700 tokens ◦ Medium (1,000 tokens): total ≈ 3,500 tokens 3) Large function payloads (comparable to one “sheet”; say ~2,000 characters ≈ 500 tokens each) • Input per call ≈ 500 tokens → 10 calls = 5,000 input tokens. • With outputs: ◦ Short (200 tokens): total ≈ 5,200 tokens ◦ Medium (1,000 tokens): total ≈ 6,000 tokens Comparison summary • Your single long prompt input is ~4,858 tokens. If your 10 function calls each send small payloads, the function approach will consume far fewer tokens. If each function call carries large payloads (hundreds of tokens), total input can exceed the single long prompt. • The dominant factor is how large the function-call payloads are and how verbose the model’s outputs are. Ten tiny function calls + a short combined output is the most token-efficient scenario; ten large calls can be more expensive than the single long prompt. Next step / offer to help • If you share the approximate character or JSON size of each function call payload and an expected total output length, I’ll compute a tighter token estimate for both approaches.

*Asks for EMI:**
        - If {emi_eligible} = no: "Sorry, is card par EMI available nahi hai. Minimum pay karke account regular karein."
        - If {emi_eligible} = yes:
          1. "EMI available hai. Interest {emi_interest_rate} lagega. Options: {emi_tenure_options}. Kaunsa chahenge?"
          2. Customer selects (e.g., "6 months") → Look up amount from {emi_tenure_options}
          3. "6 months ke liye EMI approximately [AMOUNT] banegi. Kya final karoon?"
          4. If yes: "App se activate karein. Aaj kis time tak karenge?" → Get time → Record PTP
        
        **Asks for waiver:**
        - If {waiver_eligible} = 1: "Main {waiver_amount} tak waive kar sakti hoon. Minimum pay karo, auto-apply hoga."
        - If {waiver_eligible} = 0: "Waiver available nahi hai. Minimum pay karke charges se bacho."
        - Critial Rule: waivre will be given only after the payment.
        **Customer asks for more waiver than {waiver_amount}:**
        "Maximum {waiver_amount} hi available hai. Main isse zyada nahi kar sakti. Kya ye accept karenge?"

this is the prompt snippet, as you can see, before calling i have the info if the customer is emi_eligible or is waiver_eligible, my approach was to map the tools on the basis of the data, let say the customer is emi_eligible, only then i load the emi_eligibility tool for the customer, which can let the agent know what to speak, if the customer is not eligible i do not load this tool. Is this a good approach other wise the prompt is huge and every interaction consumes 7-8k token and overall total_token=80k for a call

Thanks — this is a clear explanation and a smart question. You’re asking whether conditionally loading a tool (e.g., only load the EMI tool when

emi_eligible = yes

) is a good way to reduce token usage versus including everything in one large prompt. I’ll confirm and give concrete best practices you can apply with GPT‑4o realtime. Short answer • Yes — loading only the tools you need based on eligibility is a good and token-efficient approach. It avoids repeatedly sending large prompt snippets that aren’t relevant to the current user and can greatly reduce input tokens per interaction. Why this works • Unused prompt content still counts toward input tokens. If you include the full EMI and waiver logic every time, you pay the token cost even when half the logic isn’t needed. • Sending only the relevant prompt/tool payload keeps the input smaller and focused, and reduces total_token for the call. Practical recommendations and tips 1. Minimal control prompt ◦ Keep a small, fixed system/instruction prompt that explains agent style and the function-calling mechanism. ◦ Example: a short line that tells the model to use available tools and wait for user confirmation before final actions. 1. Conditional tool loading ◦ Only attach the EMI tool when

emi_eligible = yes

. ◦ Only attach the waiver tool when

waiver_eligible = 1

. ◦ Keep each tool’s payload concise (only the parameters the tool needs to operate). 1. Keep tool schemas small ◦ Send only keys needed for logic. Instead of including full example dialogues in the tool payload, store those templates server-side and send only a small template ID or minimal prompt text. 1. Use short function responses ◦ Prefer tools to return compact structured data (e.g., {action: "offer_emi", tenures:[6,12], rates:[...]}), not long natural-language responses. Construct the final message once using a light template on the client/agent side. 1. Avoid re-sending large static text ◦ If you have long policy text or multi-step scripts, host them server-side and reference them by ID instead of embedding the full text each call. 1. Batch or defer verbose outputs ◦ If multiple function calls are needed, keep them focused and small. Generate one final human-facing message rather than multiple long outputs from the model. 1. Estimate and monitor ◦ Use the 4 chars ≈ 1 token rule for quick estimates. Your 19,432-character prompt ≈ 4,800–5,000 tokens—so removing that when not needed matters a lot. ◦ Track actual token usage in real sessions and iterate: measure which tool payloads are the largest and trim them. Specific to your snippet • Instead of embedding the entire multi-step EMI script in the prompt every time, keep a concise EMI tool that returns available tenures, rate, and an instruction ID. Then format the agent’s messages locally using your small template and the tool output. • For waiver: send just

{waiver_eligible, waiver_amount}

and let the local template produce the messages (e.g., “Main {waiver_amount} tak waive kar sakti hoon…”). When the conditional approach might not be enough • If your logic requires the model to see historical conversation context (long transcripts), you’ll still need to manage that history size: summarize or send only the recent/relevant turns to stay efficient. Offer to help • If you want, share: ◦ Typical JSON sizes for each tool payload (characters/tokens), and ◦ Typical expected output length, and I’ll estimate token savings and suggest a concrete refactor to get your per-call tokens well below 7–8k. You’re on the right track — conditional tool loading plus compact schemas and server-side templates will likely cut your token usage substantially.

do i need to instruct the LLM which tool to use or the tool docstring enough for LLM to take decision

Thanks — great question. I’ll respond to your question about whether you must explicitly instruct the LLM which tool to use or whether a tool’s docstring is enough for the model to decide. Short answer • A clear, concise tool docstring is often enough for the LLM to choose the correct tool, but providing brief explicit guidance in the system prompt (or minimal instruction) improves reliability—especially for edge cases or when multiple tools overlap. Why docstrings can work • Tool docstrings describe each tool’s purpose and parameters. The LLM can usually map user intent to the correct tool if the docstrings are specific and non-overlapping. • When tools have clear, single-purpose responsibilities and unique parameter signatures, the model reliably picks the right tool from the docstrings alone. When you should add explicit instruction • Tools overlap in responsibility (e.g., both can respond to payment-related requests): a short system-level instruction reduces mis-selection. • The conversation context is noisy or long and the right tool depends on subtle conditions (eligibility flags, previous steps). • You must enforce strict behavior (e.g., never offer a waiver before payment). A direct rule in the system prompt guarantees adherence. • You require high accuracy for tool selection (e.g., production agent workflows). Minimal explicit guidance improves consistency. Recommended hybrid approach (practical and token-efficient) 1. Keep short, precise tool docstrings that state the tool’s single purpose, inputs, and outputs. 2. In the system prompt include 1–2 short rules: ◦ How to prefer tools (e.g., “If emi_eligible=true, use the EMI tool to fetch tenures; otherwise do not call it.”) ◦ Critical safety rules (e.g., “Do not promise waiver before payment.”) 1. Make tool schemas narrow and unambiguous so the model can match intent by signature. 2. Use eligibility flags and upstream logic to conditionally attach tools. That reduces selection ambiguity and token usage. 3. Prefer the model returning structured outputs from tools (compact JSON) and do final wording locally. Example (conceptual) • Tool docstring: “EMI_TOOL: returns available EMI tenures and monthly amounts when emi_eligible=true. Inputs: customer_id, loan_amount. Output: {tenures:[], amounts:[]}. Do not call if emi_eligible=false.” • System prompt (short): “Use tools only when relevant. Check eligibility flags first. Never offer a waiver before confirming payment.” When you can skip explicit instruction • If your tool set is small, clearly named, and every tool has a single, obvious use, you can rely mainly on docstrings and let the model decide. If you’d like I can: • Draft a short example system prompt and compact docstrings for your EMI and waiver tools, or • Estimate token savings for attaching tools conditionally vs always including full tool descriptions. Which would help you more: an example prompt + docstrings or a token-estimate for your current tool payload sizes?

should i store the customer data in UserData as for below:

*CURRENT CUSTOMER SNAPSHOT**
      - Customer: {customer_name}
      - Outstanding: {outstanding_balance}
      - Minimum due: {minimum_due_amount}
      - Due: {due_date}
      - Days past due: {days_past_due}
      - EMI eligible: {emi_eligible}
      - EMI interest rate: {emi_interest_rate}
      - EMI options: {emi_tenure_options}
      - Waiver eligible: {waiver_eligible}
      - Waiver amount: {waiver_amount}

Thanks — this is a great, practical question. I’m responding to your ask about whether to store the customer snapshot you showed in

UserData

, and how to do that in a way that saves tokens and keeps agent logic reliable. Short answer • Yes—store the dynamic customer snapshot in

UserData

. But don’t embed the full raw snapshot into every prompt. Keep

UserData

as the canonical per-session state and send only the minimal, relevant pieces into the LLM context or tools when needed. Why this approach • Keeping customer fields in

UserData

makes them available to agents and tools without repeatedly re-sending the full snapshot as prompt text (which drives up token usage). •

UserData

is designed for arbitrary per-session state and can be accessed in tools via the run context, making it efficient and consistent. Practical recommendations 1. Define a compact schema ◦ Use a typed schema or dataclass (e.g., Python/TypeScript) for `UserData`: ▪︎ customer_name ▪︎ outstanding_balance ▪︎ minimum_due_amount ▪︎ due_date ▪︎ days_past_due ▪︎ emi_eligible (bool) ▪︎ emi_interest_rate ▪︎ emi_tenure_options (compact structure) ▪︎ waiver_eligible (bool) ▪︎ waiver_amount ◦ Keep values in machine-readable formats (numbers, booleans, ISO dates, arrays). 1. Condition the LLM/context on flags, not full dumps ◦ Instead of inserting the full snapshot into prompts, check flags first and only inject what’s required: ▪︎ If

emi_eligible = true

, attach a small EMI payload or call the EMI tool. ▪︎ If

waiver_eligible = true

, attach only

{waiver_amount}

. ◦ This prevents unnecessary token usage when many fields are irrelevant. 1. Keep tool payloads minimal ◦ Tool inputs should be compact (e.g.,

{customer_id, emi_eligible, principal}

or

{waiver_eligible, waiver_amount}

). ◦ Tool outputs should be structured and short (JSON with only necessary keys like

{tenures:[6,12], amounts:[1234,678]}

). 1. Host long static scripts/templates server-side ◦ Store multi-step scripts or long policy text (like full agent dialogues) on your server and reference them by ID. Send only a short template ID or one-line instruction to the LLM, then render the user-facing text locally using the structured tool output. 1. Use lifecycle hooks to manage data loading ◦ Load static or session-level customer data into

UserData

at session startup (or in a prewarm step). ◦ For large or rarely-used fields, fetch them on demand (or inject via RAG) just before they’re needed rather than at every turn. 1. Summarize long histories ◦ If you must include conversation history, summarize it or send only the most recent relevant turns to reduce tokens. 1. Safety & correctness rules in system prompt ◦ Keep short system rules for critical constraints (e.g., “Do not promise waiver before payment”). Store the rule once in system context; don’t repeat it with every user turn. Concrete example flow 1. Session start: load

UserData

with the compact schema. 2. On user request: ◦ Check

UserData.emi_eligible

. ◦ If true: attach the EMI tool (or call it) with

{customer_id, principal}

→ get

{tenures, amounts}

. ◦ Locally format the agent message using the short template and tool output. ◦ If false: do not attach EMI tool; use a short waiver or default flow. 1. For waiver: only attach

{waiver_eligible, waiver_amount}

and apply the system rule that waiver is post-payment. When to still include fields in prompt • If the model must generate nuanced language that depends on multiple fields at once (rare), send only those few fields—not the full snapshot. • If you need the LLM to reason over a combination of many attributes repeatedly within a single turn, consider a short summarized representation rather than the full raw data. If you want, I can: • Draft a compact

UserData

dataclass/schema example for your stack (Python/TypeScript). • Show an example minimal tool schema + a one-line system prompt to enforce the waiver rule. • Estimate token savings if you share typical sizes (characters) of your current prompt vs. compact payloads. Which would you like next? A schema example or a token-savings estimate? Sources: Agent session | LiveKit Documentation | Agents and handoffs | LiveKit Documentation | External data and RAG | LiveKit Documentation

do steps 1 and 2

Thanks — I’ll deliver both items you asked for: 1) A compact

UserData

schema (Python dataclass and TypeScript interface) you can use as the canonical per-session state. 2) A rough token‑savings estimate comparing your current full snapshot prompt vs. a compact/conditional approach. 1) Compact UserData schema Python (dataclass)

from dataclasses import dataclass
from typing import List, Optional
from datetime import date

@dataclass
class EMIOption:
    tenure_months: int
    monthly_amount: int  # or float, in smallest currency unit if preferred

@dataclass
class UserData:
    customer_id: str
    customer_name: Optional[str] = None

    # balances
    outstanding_balance: float
    minimum_due_amount: float
    due_date: Optional[date] = None
    days_past_due: Optional[int] = None

    # EMI
    emi_eligible: bool = False
    emi_interest_rate: Optional[float] = None  # e.g., 12.5
    emi_tenure_options: Optional[List[EMIOption]] = None

    # Waiver
    waiver_eligible: bool = False
    waiver_amount: Optional[float] = None

    # metadata (short)
    last_update_iso: Optional[str] = None  # ISO timestamp for freshness checks

TypeScript (interface)

export interface EMIOption {
  tenure_months: number;
  monthly_amount: number; // cents or smallest currency unit recommended
}

export interface UserData {
  customerId: string;
  customerName?: string;

  outstandingBalance: number;
  minimumDueAmount: number;
  dueDate?: string; // ISO date string
  daysPastDue?: number;

  emiEligible: boolean;
  emiInterestRate?: number;
  emiTenureOptions?: EMIOption[];

  waiverEligible: boolean;
  waiverAmount?: number;

  lastUpdateIso?: string;
}

Design notes • Keep numeric values as numbers (use smallest currency unit if you want to avoid floats). • Use booleans for quick gating (emiEligible / waiverEligible). • Keep

emiTenureOptions

compact: only tenure and monthly_amount (no verbose descriptions). • Include a freshness timestamp so you can decide when to re-fetch large fields. 2) Rough token‑savings estimate Assumptions from your earlier data: • Full snapshot you showed: 19,432 characters ≈ 4,858 tokens (rule of thumb: ~4 chars/token). • You reported per-interaction totals currently around 7–8k tokens (likely because you send the large snapshot + other context + outputs). Compact approach (what we’ll send into the LLM) • Minimal system prompt / rules: ~300–500 characters ≈ 75–125 tokens. • Minimal injected fields per turn: only the flags and the few needed numeric values (example payload below). Example minimal payload (characters ~200) • JSON like: {"customerId":"123","emiEligible":true,"waiverEligible":false,"outstandingBalance":12345,"minimumDue":500} • Estimated tokens: ~50 tokens Scenarios and totals (input + a typical model reply) 1. Current way (full snapshot): ◦ Input ≈ 4,858 tokens ◦ Model output (medium) ≈ 1,000 tokens ◦ Total ≈ 5,858 tokens (matches your 7–8k when combined with other context/history) 1. Compact / conditional tool approach: ◦ System + minimal payload input ≈ 125 + 50 = 175 tokens ◦ Model output (same medium) ≈ 1,000 tokens ◦ Total ≈ 1,175 tokens 1. Conditional with function/tool calls and compact tool responses: ◦ System + minimal payload input ≈ 175 tokens ◦ Tool call (small JSON request) ≈ 50–200 tokens depending on fields ◦ Tool response (structured) ≈ 30–150 tokens ◦ Final formatted output by LLM ≈ 200–500 tokens ◦ Total typical ≈ 455–1,025 tokens Estimated savings • Going from the full snapshot to the compact approach typically reduces input tokens from ~4,858 to ~175 (a ~96% reduction in input size). • For a medium reply (1,000 tokens), total tokens drop from ~5,858 to ~1,175 — roughly 4–5x reduction in total token consumption. • Using compact tool calls and rendering the final message locally can reduce totals further (often by another 20–50% depending on how much you offload). Practical implementation tips • Gate tool attachment by flags: only attach EMI tool if

emiEligible

is true; only attach waiver tool if

waiverEligible

is true. • Keep tool schema small and return structured JSON; render final PHRASES locally or with a short formatting prompt. • Avoid re-sending full customer snapshots; store them in

UserData

and fetch selectively or summarize. • Monitor actual token usage in production and iterate. The rule-of-thumb estimation is good for planning but measure real traffic to fine-tune. If you want, I can: • Draft the minimal system prompt + example tool schema and a short rendering template for the EMI flow, OR • Take one real example payload you currently send and compute exact token counts (if you paste the actual text). Which would you prefer?

livekit also has UserData, we can use that

Great — using LiveKit’s

UserData

is exactly the right place for this snapshot. I’ll confirm how to map the schema into LiveKit’s

UserData

, show a minimal example of how to load/use it in the LiveKit agent flow, and give a few concrete implementation notes so you get the token savings we discussed. What to store in LiveKit UserData • Store the compact schema fields (the dataclass/interface from earlier) in LiveKit’s

UserData

. Keep data machine-readable (numbers, booleans, ISO dates) and include a freshness timestamp. • Avoid stuffing long human-readable scripts or policies into

UserData

. Store only the values needed for decision logic and lookups. Minimal LiveKit UserData example (conceptual) • Use the same compact fields as before. Persist this object in LiveKit

UserData

at session start or when you fetch the customer snapshot. Example (JSON-like representation to save into LiveKit UserData) { "customerId": "12345", "customerName": "Asha Sharma", "outstandingBalance": 12345.50, "minimumDueAmount": 500.00, "dueDate": "2026-03-25", "daysPastDue": 10, "emiEligible": true, "emiInterestRate": 12.5, "emiTenureOptions": [{"tenure_months":6,"monthly_amount":2100}], "waiverEligible": false, "waiverAmount": 0, "lastUpdateIso": "2026-03-12T100000Z" } How to use this in the LiveKit agent flow (recommended pattern) 1. Session startup / prewarm ◦ Load the customer snapshot into

UserData

once (or when the session is created). This keeps the snapshot out of the chat prompt while making it available to tools and nodes via the RunContext. 1. Turn handling (on user request) ◦ Check flags in

UserData

first (e.g.,

emiEligible

,

waiverEligible

) to decide which tools to attach or which flows to run. ◦ If

emiEligible

is true, attach or call the EMI tool with a very small payload (e.g.,

{customerId, outstandingBalance}

). ◦ If

waiverEligible

is true, attach or call the waiver tool with

{customerId, waiverAmount}

. ◦ If neither flag applies, do not attach those tools. 1. Tool input/output policy ◦ Inputs: keep them minimal — IDs and the small numeric fields needed for calculations. ◦ Outputs: tools should return compact JSON (e.g.,

{tenures: [6], amounts: [2100]}

). ◦ Use the agent or local code to format natural-language responses using those structured outputs (minimizes LLM token usage). 1. Injecting minimal context into LLM ◦ When you do call the LLM, pass a tiny context snippet if needed (e.g., system rules + small JSON of relevant fields). ◦ Example minimal injection:

"emiEligible": true, "emiTenureOptions": [{"tenure":6,"amount":2100}]

rather than the full snapshot text. Practical LiveKit-specific tips • Use LiveKit

UserData

for session state so you don’t re-send the full snapshot each turn. • Load static templates or long scripts server-side and reference them by ID in the agent (send only the ID to the LLM). • For large or seldom-used fields, fetch on demand and populate

UserData

only when needed. • Use

lastUpdateIso

to decide when to refresh

UserData

instead of always re-fetching on each turn. Safety and rule enforcement • Keep critical rules (e.g., “waiver is applied only after payment”) in the system-level instructions once — not repeated each turn — and enforce them in your agent logic using

UserData

flags. If you want, I can: • Provide a short LiveKit agent pseudocode snippet showing how to load UserData at prewarm, gate tool attachment by flags, and render final messages locally; or • Draft a minimal system prompt + two compact tool docstrings (EMI and Waiver) you can paste into LiveKit tool definitions. Which would you like next? A pseudocode example or the tool docstrings?

a pusedocode example

Thanks — below is a concise LiveKit agent pseudocode example showing how to: • Load

UserData

at session startup (prewarm). • Gate tool attachment/calls by eligibility flags (

emiEligible

,

waiverEligible

). • Call small tools that return compact JSON. • Render the final user-facing message locally using templates (minimizes tokens sent to the LLM). This is language-agnostic pseudocode; adapt it to your LiveKit SDK and runtime (Python/TypeScript). Pseudocode — LiveKit agent flow 1. Session startup / prewarm

onSessionStart(sessionId, customerId):
    # Fetch full customer snapshot once from your DB / backend
    snapshot = fetchCustomerSnapshot(customerId)

    # Map and store compact snapshot into LiveKit UserData
    userData = {
        customerId: snapshot.id,
        customerName: snapshot.name,
        outstandingBalance: snapshot.outstanding_balance,
        minimumDueAmount: snapshot.minimum_due,
        dueDate: snapshot.due_date_iso,
        daysPastDue: snapshot.days_past_due,
        emiEligible: snapshot.emi_eligible,        # boolean
        emiInterestRate: snapshot.emi_interest_rate,
        emiTenureOptions: compactify(snapshot.emi_options),  # [{tenure_months, monthly_amount}, ...]
        waiverEligible: snapshot.waiver_eligible,  # boolean
        waiverAmount: snapshot.waiver_amount,
        lastUpdateIso: now_iso()
    }

    LiveKit.UserData.set(sessionId, userData)

1. On each user turn (minimal context + gating)

onUserMessage(sessionId, userMessage):
    userData = LiveKit.UserData.get(sessionId)

    # Quick local pre-checks (no LLM cost)
    if userMessage.intent == "ask_emi":
        if not userData.emiEligible:
            # Local template reply — no tool, no heavy prompt
            reply = "Sorry, EMI available nahi hai. Minimum pay karke account regular karein."
            sendReply(sessionId, reply)
            return

        # EMI eligible -> proceed with small EMI tool call
        emiRequest = {
            customerId: userData.customerId,
            outstandingBalance: userData.outstandingBalance
        }
        # call compact EMI tool (keeps tokens small)
        emiResult = callTool("EMI_TOOL", emiRequest)
        # emiResult example: {tenures: [6, 12], amounts: [2100, 1100], rate: 12.5}
        # Render message locally using a short template
        reply = formatEMIOptionsTemplate(emiResult)
        sendReply(sessionId, reply)
        return

    if userMessage.intent == "ask_waiver":
        if not userData.waiverEligible:
            reply = "Waiver available nahi hai. Minimum pay karke charges se bacho."
            sendReply(sessionId, reply)
            return

        # Waiver eligible -> short confirmation flow
        reply = "Main {waiverAmount} tak waive kar sakti hoon. Minimum pay karo, auto-apply hoga.".format(waiverAmount=userData.waiverAmount)
        sendReply(sessionId, reply)
        return

    # For other intents or when LLM is required for language generation:
    # Build a minimal LLM context (system rules + only necessary fields)
    llmContext = {
        systemPrompt: "You are a polite collections agent. Do NOT promise waiver before payment.",
        userMessage: userMessage.text,
        relevantData: pickRelevant(userData, userMessage)  # e.g., {emiEligible, waiverEligible, outstandingBalance}
    }

    llmResponse = callLLM(llmContext)  # keep this payload minimal
    sendReply(sessionId, llmResponse.text)

1. Example compact tool definitions (docstrings + schema)

# EMI_TOOL docstring:
# "EMI_TOOL: Given {customerId, outstandingBalance}, return available tenures and monthly amounts.
#  Input: {customerId: string, outstandingBalance: number}
#  Output: {tenures: number[], amounts: number[], rate: number}
#
#  Do not call if emiEligible=false."

# WAIVER_TOOL docstring:
# "WAIVER_TOOL: Given {customerId}, return waiver eligibility and max waiver amount.
#  Input: {customerId: string}
#  Output: {waiverEligible: boolean, waiverAmount: number}
#
#  Use only when waiverEligible may be stale or needs verification."

1. Templates / local rendering (avoid sending long scripted text to LLM)

formatEMIOptionsTemplate(emiResult):
    # Small, deterministic formatting done server-side
    options = zip(emiResult.tenures, emiResult.amounts)
    lines = ["EMI available hai. Interest {rate}% lagega. Options:".format(rate=emiResult.rate)]
    for tenure, amount in options:
        lines.append("- {tenure} months: approx {amount} per month".format(tenure=tenure, amount=amount))
    lines.append("Kaunsa chahenge?")
    return joinLines(lines)

1. State updates and persistence

# If user confirms EMI or sets PTP, update UserData and your DB
onUserConfirmEMI(sessionId, chosenTenure):
    # record activation steps, PTP time, etc.
    updateBackendEMIActivation(userData.customerId, chosenTenure)
    userData.lastUpdateIso = now_iso()
    LiveKit.UserData.set(sessionId, userData)

Notes and best practices • Always prefer local checks on

UserData

flags before attaching tools or hitting the LLM. • Keep tool inputs/outputs minimal and structured (JSON). That reduces token size dramatically. • Store long scripts, policy texts, or examples on your server and reference them by ID instead of embedding them in prompts. • Use

lastUpdateIso

to refresh large fields only when stale. • Keep critical safety rules in the system prompt once (e.g., no waiver before payment); enforce in code as well. If you want, I can convert this pseudocode into a runnable example in your language of choice (Python or TypeScript) tied to the LiveKit SDK you use. Which language would you prefer?