https://livekit.io logo
Join Slack
Powered by
How to check the previous speech is from agent or ...
# ask-ai
l
How to check the previous speech is from agent or caller?
t
Thanks for reaching out, Rajadurai. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding to your question about how to tell whether a previous speech segment came from an agent or a caller. Short answer: use the transcription/text-stream metadata and the event source. Caller speech is emitted as transcription events (or appears on the transcription text stream with a transcribed track id), while agent speech is produced by your agent code (so it won’t have that transcribed-track metadata). How to determine this reliably • Listen for transcription events: ◦ The 
user_input_transcribed
event is emitted for non-agent participants (callers). If you receive that event, the speech is from a caller. • Inspect text-stream message attributes: ◦ Messages published to the 
lk.transcription
topic include a 
sender
identity. ◦ If the text message includes the 
lk.transcribed_track_id
attribute, it was produced from transcribed audio (caller). ◦ If there is no 
lk.transcribed_track_id
but you still see a message in 
lk.transcription
, it’s likely agent-originated text (for example, text your agent published). • Track agent outputs in your code: ◦ Agent speech you trigger via 
session.say()
or 
session.generate_reply()
is agent-originated — you control those calls, so you can mark/store them as agent speech when emitting them. • Use participant kind as a filter: ◦ Participants marked as 
Agent
are typically ignored for transcribed audio; transcription events usually come from 
User
or 
SIP
participants. If a transcribed event’s participant kind is not 
Agent
, the speaker is a caller. Simple example logic 1. On 
user_input_transcribed
-> treat as caller speech. 2. When you call 
session.say(...)
or 
session.generate_reply(...)
-> mark that output as agent speech. 3. When handling text-stream messages, check `lk.transcribed_track_id`: ◦ Present = caller (transcribed audio) ◦ Absent = likely agent-originated text Would you like a short code example for a specific SDK or language (JavaScript, Python, or another)? That will let me show exact event names and attribute checks for your environment. Sources: examples/frontdesk | livekit/livekit_composite | DeepWiki | Text and transcriptions | LiveKit Documentation | Agent speech and audio | LiveKit Documentation | sip | livekit/livekit_composite | DeepWiki
l
give a python code to check the previous output if previous output from agent skip generate_reply() else generate_reply()
t
Thanks — I’m responding to your request for a Python example that checks whether the previous output came from the agent, and only calls 
generate_reply()
when the previous output was from a caller. Below is a concise example pattern you can adapt. It keeps a small state flag (
last_was_agent
) that updates when your code sends agent output (via 
session.say()
or 
session.generate_reply()
) and also updates when a transcription event arrives (caller speech). Before calling 
generate_reply()
, it checks the flag and skips if the last output was already from the agent.
Copy code
# Pseudocode-style example — adapt to your LiveKit session/event APIs
# Replace event names and method calls with those from your SDK.

class ConversationManager:
    def __init__(self, session):
        self.session = session
        # True if the most recent output was produced by the agent
        self.last_was_agent = False

        # Hook event listeners
        self.session.on("user_input_transcribed", self.on_user_transcribed)
        self.session.on("text_stream_message", self.on_text_stream_message)

    def on_user_transcribed(self, event):
        # This event fires for caller speech (transcribed audio).
        # Mark last output as NOT agent because caller just spoke.
        self.last_was_agent = False
        print("Caller spoke (transcription):", event.transcript)

        # Decide whether to generate a reply now:
        self.maybe_generate_reply(event)

    def on_text_stream_message(self, msg):
        # If you receive text messages that represent agent output,
        # you can use the presence/absence of `lk.transcribed_track_id`
        # or the `sender` identity to decide origin.
        # Example: agent-produced text won't have `lk.transcribed_track_id`.
        attrs = getattr(msg, "attributes", {}) or {}
        if "lk.transcribed_track_id" not in attrs and msg.sender == "agent":
            # Mark that the agent sent this message
            self.last_was_agent = True
            print("Agent produced text:", msg.text)
        else:
            # This is likely caller-transcribed text
            self.last_was_agent = False

    def maybe_generate_reply(self, trigger_event):
        # Only generate a reply when the previous output was not already the agent.
        if self.last_was_agent:
            print("Skipping generate_reply(): previous output was from agent.")
            return

        # Otherwise, call generate_reply() and mark the last output as agent.
        reply = self.session.generate_reply(trigger_event)  # adapt params as needed
        self.last_was_agent = True
        print("Called generate_reply():", reply)

    # If you explicitly send agent speech through code elsewhere, set flag:
    def send_agent_text(self, text):
        self.session.say(text)
        self.last_was_agent = True

# Usage example:
# session = ...  (your LiveKit session object)
# mgr = ConversationManager(session)
Key points to adapt to your environment • Use 
user_input_transcribed
events to detect caller speech — those events are emitted for non-agent participants. • For text-stream messages on 
lk.transcription
, check `attributes["lk.transcribed_track_id"]`: ◦ If present → transcribed audio (caller). ◦ If absent and 
sender
identifies your agent → agent-originated text. • Every place in your code where you explicitly produce agent output (
session.say()
, 
generate_reply()
, etc.) should set 
last_was_agent = True
. • When you detect caller speech (transcription event or transcribed text), set 
last_was_agent = False
before deciding to call 
generate_reply()
. Would you like this adapted into a runnable example for a specific LiveKit Python SDK (showing exact event registration and message attribute access)? If so, tell me which SDK/client you’re using or paste the event names you have, and I’ll tailor the snippet.
3 Views
Open in Slack
PreviousNext
Powered by