https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# ask-ai
s
This message was deleted.
✅ 1
t
Thanks for reaching out, Aleh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — this is a great, specific question. I’m responding to your ask about the three approaches (system prompt updates, the on_user_turn_completed hook, and a function tool) and whether injecting the date in on_user_turn_completed is the right choice and how long that context persists. Short recommendation • Use the on_user_turn_completed hook to inject a Python-computed current date/time message into the chat context immediately before LLM inference as the primary solution. • Optionally add a lightweight function tool (e.g., get_date_info) as a verification/fallback the LLM can call for critical confirmations. Why on_user_turn_completed is the best fit • Freshness: The hook runs after the user turn is finished and immediately before the LLM generates its reply, so you can compute the date/time right at the moment the model needs it. • Prominence: Adding the date/time as a chat message makes it highly visible to the LLM (rather than buried in instruction metadata). • Simplicity: It avoids depending on the model to call a tool, and it does not require frequent system-prompt churn. • Practical: This aligns with the pattern of injecting dynamic external data into the conversation context just-in-time. Notes on the other two approaches • System prompt (update_instructions) ◦ Not ideal for highly dynamic values. Updating the system prompt can work, but it’s less reliable for per-inference freshness and can make the system instructions noisy if you frequently inject dynamic content. ◦ Frequent instruction updates risk confusing the LLM about role vs. dynamic facts. • Function tool (get_date_info) ◦ Good as a verification step: define a small tool that returns the computed day-of-week/date for a proposed ISO date. ◦ Downside: the LLM must choose to call it. If it’s already confidently hallucinating, it may skip the call. ◦ Best used together with on_user_turn_completed as a safety check before finalizing output (e.g., bot computes a date, calls the tool to confirm, then speaks). Behavior of on_user_turn_completed and persistence • Messages you add into the ChatContext via on_user_turn_completed become part of the chat history and persist across subsequent turns by default. • If you want the injected message to be transient (apply only to the immediate response), you can avoid persisting it to the long-term chat history — but persistence is generally helpful here to prevent repeating the same hallucination in later turns. • In practice: inject a clear line such as: ◦ "Current date and time: Saturday, February 14, 2026 — Use this exact date when resolving phrases like 'next Friday' or 'in two weeks.'" Practical implementation tips 1. Compute and format a single authoritative time string in Python (include timezone). 2. Insert it as an assistant/system-style chat message in on_user_turn_completed right before inference. 3. Make the instruction explicit about how the LLM should use it (e.g., "Always use this current date/time to resolve relative dates; if you propose a calendar date, verify it matches this baseline"). 4. Optionally implement a small 
get_date_info(iso_date)
function tool: ◦ Use it to validate dates the LLM intends to speak aloud (call it automatically in your response-generation pipeline if you control that flow). ◦ Or prompt the LLM to call it before committing to any spoken scheduled date. Example (conceptual) 1. on_user_turn_completed computes: ◦ "Current date/time: Saturday, February 14, 2026, 03:45 PM PST." 1. Add message to ChatContext: ◦ role: assistant ◦ content: "Current date/time: ... Use this for scheduling calculations." 1. Let the LLM generate a reply; if the reply contains a concrete date, optionally invoke 
get_date_info
to confirm before the bot speaks. If you want, I can: • Provide a short code snippet for using on_user_turn_completed to insert the date/time with correct formatting and timezone. • Sketch a minimal 
get_date_info
function-tool implementation and show how to call it for verification. Which would you prefer? Sources: Building an Automated IVR Menu Caller | LiveKit Documentation | Amazon Nova Sonic integration guide | LiveKit Documentation | Function tool definition | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | External data and RAG | LiveKit Documentation, with interactive elements
b
on_user_turn_completed is unsupported in my realtime setup
t
Thanks — that clarification helps. Since 
on_user_turn_completed
is unsupported in your realtime setup, here are practical alternatives you can use to ensure the LLM gets an accurate, Python-computed current date/time for scheduling and avoid day-of-week hallucinations. Short recommendation • Use a combined approach: (A) keep an authoritative current datetime in the system instructions updated on a short interval, and (B) provide a small function tool (e.g., 
get_current_datetime
/ 
get_date_info
) that the model is strongly encouraged — or forced by your server logic — to call for any concrete date the bot intends to speak. This balances freshness with reliability in realtime models. Why this combination works • Realtime models don’t expose the post-turn hook, so you need server-side mechanisms to push dynamic context into the LLM’s inputs. Periodically updating the system instructions gives the model a recent baseline; the function tool gives a deterministic verification step that returns the exact day-of-week for any ISO date. • The function tool covers the failure mode where the model confidently (but incorrectly) converts relative dates — the bot or the model can verify before speaking. Implementation options and tradeoffs 1) System instructions updated periodically (recommended baseline) • What to do: ◦ Run a background task that updates the agent’s system instructions with a single authoritative timestamp (include timezone). ◦ Update frequency depends on your needs: every minute for high precision, every 5–15 minutes for most scheduling use cases. • Pros: ◦ Simple and guaranteed to be present in each request. • Cons: ◦ Slightly stale between updates; frequent updates add overhead. • Tip: Keep the content minimal and prescriptive, e.g.: ◦ "Current date/time (UTC): 2026-03-23T154500Z. Use this to resolve relative dates like 'next Friday'." 2) Function tool for deterministic verification (highly recommended) • What to do: ◦ Implement a 
get_current_datetime()
and/or 
get_date_info(iso_date)
function tool in Python that returns exact date and day-of-week in a clear format. ◦ Encourage the LLM to call it for scheduling language, or have your server automatically call it when the LLM proposes a concrete date. • Pros: ◦ Deterministic, authoritative, and prevents the model from speaking incorrect days. ◦ Small and inexpensive to run. • Cons: ◦ If left purely to LLM discretion, model might not call it; mitigate by prompting policies or server-side verification. • Tip: If you control the response pipeline, run the tool automatically on any candidate date the LLM outputs and replace/confirm the date before TTS. 3) Initial context injection at session start (only for low-dynamism cases) • What to do: ◦ Add the session start time to the system or chat context when the session begins. • Pros: ◦ Easy to implement. • Cons: ◦ Only accurate for the session start; not good for long sessions where date may change. 4) Server-side parsing / verification step (defensive) • What to do: ◦ After the LLM generates a reply but before TTS, parse any proposed calendar dates and verify them with your Python date functions. If mismatch, either correct it or prompt the model to re-confirm. • Pros: ◦ Last-line defense that catches hallucinations even if model ignores tools. • Cons: ◦ Adds a verification step and latency. Concrete workflow I recommend 1. Periodically update system instructions with a concise, authoritative timestamp (e.g., every 1–5 minutes). 2. Expose a 
get_date_info(iso_date)
function tool that returns day-of-week and formatted date. 3. In your response pipeline: ◦ If the LLM outputs a concrete calendar date, automatically call 
get_date_info
to confirm the day-of-week before sending to TTS. ◦ If mismatch, correct the text or re-run the LLM with the verified date injected in the prompt. 1. Make the system instruction explicit: "Always use the current date/time in the system instruction for schedule math. Before speaking any specific date, verify it using the get_date_info tool." Example verification pseudocode (conceptual) • LLM generates candidate reply -> parse candidate for dates -> for each date call get_date_info(iso) -> if result differs from reply, replace or re-run LLM with verified date in prompt -> send to TTS. If you want, I can: • Provide a short Python snippet for a minimal 
get_date_info
function tool and an example of how to auto-verify LLM output before TTS. • Help pick an update cadence for system instructions based on your typical call length and latency constraints. Which would you like next — the code snippet for the function tool, or the server-side verification example? Sources: Pipeline nodes and hooks | LiveKit Documentation | Build a salary negotiation coach with a LemonSlice avatar | LiveKit | Function tool definition | LiveKit Documentation | External data and RAG | LiveKit Documentation | Realtime models overview | LiveKit Documentation, with interactive elements
2 Views
Open in Slack
PreviousNext
Powered by