https://livekit.com/blog/observer-pattern-voice-agent-guardrails Can this be supported in Realtime models?

Thanks for reaching out, Samuel. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I use gpt-realtime model, it emits lk.transcription, can't we use that?

Thanks — good detail. You're right that

gpt-realtime

emits

lk.transcription

. That helps, but there are important caveats. •

lk.transcription

can provide interim or final transcript pieces you can listen to, so you can use it to detect policy violations earlier than waiting for the agent response. • However, timing and completeness differ from the classic

conversation_item_added

flow: transcripts may arrive fragmented, delayed, or as partial interim results, so your observer logic must handle incremental updates and deduplication. • If you rely on a single, stable event boundary (like the full user turn delivered by

conversation_item_added

),

lk.transcription

is not an exact drop‑in — you’ll need to stitch interim chunks into coherent utterances before running evaluations, or run rolling evaluations tolerant of partial text. • Practical options: ◦ Use

lk.transcription

and buffer/reassemble interim segments into complete utterances, then call your evaluator and, when needed, inject guardrail instructions via

update_chat_ctx

. ◦ Or keep a hybrid approach (external STT or text modality) if you need strict, turn‑level events and predictable ordering. If you want, tell me: • whether you get interim transcripts or only final

lk.transcription

events, and • whether you need immediate micro‑interventions (during speech) or turn‑level interventions (after speech ends). I’ll outline a buffering pattern or sample event handling code for your case. , with interactive elements

gpt realtime emits:

conversation_item_added.

so it should be possible. check it out

You're right — gpt-realtime emits

conversation_item_added

, so the observer-pattern guardrails are supported with that model. What this means and what to watch for: • Use

conversation_item_added

(filter

role == "user"

) to trigger your background evaluator once the user turn is committed to the convo history. • You can also listen to

lk.transcription

for interim pieces, but treat them as partial fragments: buffer/stitch or run rolling evaluations tolerant of partial text. • When a violation is detected, call

update_chat_ctx

to inject system instructions into the active agent; the agent will apply them on its next turn. • Pay attention to deduplication, ordering, and whether you need micro‑interventions (during speech) versus turn‑level interventions (after speech ends). Would you like a short example of event handling (buffering vs. turn‑level) for your implementation? Sources: How to Build a Background Observer for Voice AI Guardrails | LiveKit | Text and transcriptions | LiveKit Documentation, with interactive elements

am gonna use it for on-the-fly lead scorer instead of guardrails. so is it possible?

Yes — you can use the observer pattern with

gpt-realtime

for an on‑the‑fly lead scorer. How to do it: • Listen for

conversation_item_added

(filter

role == "user"

) to get committed user turns for turn‑level scoring. • Optionally also consume

lk.transcription

for interim scoring or to build rolling scores; buffer/stitch interim fragments before finalizing a turn. • Send the transcript (or incremental text) to your scorer (a separate LLM or scoring service) and compute lead score or attributes. • Inject results into the session (e.g., via

update_chat_ctx

, metadata, or your backend) so downstream systems or agents can act on the score. • Handle deduplication, ordering, partial transcripts, and decide whether scores should update mid‑utterance or only after final turn. If you want, I can: • Provide a short event‑handler example (buffering vs turn‑level), or • Suggest a minimal JSON schema for score metadata you can attach to conversations. Which would you prefer?, with interactive elements

so for lead scorer- only user role is needed or agent role to ?

Good question — you should monitor both, but prioritize user messages. • Primary source: use

conversation_item_added

events where

role == "user"

for the core lead score (user intent, qualifications, contact info, objections). • Why include agent messages: agent prompts, questions, and confirmations change context and can affect scoring (e.g., agent qualification questions eliciting new info, or agent objections lowering lead quality). Also avoid double‑counting when agent repeats user info. • Practical pattern: a. Compute an initial score from the user turn when it’s committed. b. Recompute or adjust the score after agent turns if the agent elicited new info or clarified details. c. Optionally emit two outputs: a turn‑level provisional score (user turn) and a session‑level score that updates after agent interactions. • Implementation tips: deduplicate entities across turns, use timestamps to order updates, and flag whether a score is provisional or confirmed. Would you like a short example handler (pseudocode) for turn‑level vs. session‑level scoring? , with interactive elements