https://livekit.io logo
Join Slack
Powered by
If user hangs up just after saying something, the ...
# ask-ai
l
If user hangs up just after saying something, the last speech doesn't come in the transcript, as we are deleting the room. How to go about this
t
Thanks for reaching out, Deepak. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I'm here to help you make sure the last speech is captured perfectly in the transcript before you delete a room. Let's walk through these steps together: • Wait for Final Transcripts: It's important to wait until all final transcription events come through before deleting the room. You'll know they're final when you see the attribute 
ATTRIBUTE_TRANSCRIPTION_FINAL
set to 
true
. • Flush Transcription Output: Using the 
flush()
method in the transcription output classes is a great way to mark the transcription as final and ensure the last bit of text is published. • Bind to Session Close Events: Try using session 
close
events to trigger cleanup actions. For example, you can delay deleting the room until the session is fully closed and all transcripts are finalized. • Wait for Pending Tasks: Make sure all pending tasks, like transcription flushing, are fully completed before you proceed with deleting the room. • Delete Room After Participant Leaves: It’s best to delete the room only after the human participant has left and all transcription processes are finalized. Following these steps will help you capture every last word in the transcript before the room is deleted. If you have any questions or need further help, feel free to ask—I’m here for you! Sources: examples/other/translation/multi-user-translator.py | agents/src/voice/audio_recognition.ts
l
how to use this ATTRIBUTE_TRANSCRIPTION_FINAL in code
t
To help you use 
ATTRIBUTE_TRANSCRIPTION_FINAL
in your code, here are some friendly examples you can follow: • Python: You can set the attribute in the 
attributes
dictionary when publishing transcription data. For instance:
Copy code
attributes = {
    ATTRIBUTE_TRANSCRIPTION_FINAL: "false",  # for interim updates
    ATTRIBUTE_TRANSCRIPTION_TRACK_ID: self.track.sid,
}
writer = await self.room.local_participant.stream_text(
    topic=TOPIC_TRANSCRIPTION,
    sender_identity=self.participant_identity,
    attributes=attributes,
)

# When transcription is complete
final_attrs = {
    ATTRIBUTE_TRANSCRIPTION_FINAL: "true",
}
await writer.aclose(attributes=final_attrs)
• TypeScript: Here’s how you can check the attribute to see if a transcription is final:
Copy code
room.registerTextStreamHandler("lk.transcription", (reader, participant) => {
  const isFinal = reader.info.attributes.get('lk.transcription_final') === 'true';
  reader.on('text_received', (chunk) => {
    console.log({ text: chunk, isFinal });
  });
});
• Swift: You can use the attribute within a transcription handler like this:
Copy code
let isFinal = reader.info.attributes["lk.transcription_final"] == "true"
if isFinal {
    // Handle finalized transcription
}
• Kotlin: Here’s a simple way to check the attribute in a transcription handler:
Copy code
val isFinal = streamInfo.attributes["lk.transcription.final"]?.toBoolean() == true
I hope these examples make it easier for you to mark or check if a transcription is final. If anything feels unclear or if you want to dive deeper, just let me know—I’m here to help! Sources: livekit-agents/livekit/agents/types.py | Sources/LiveKit/Types/Attributes/AttributeTypings.swift | lib/src/types/attribute_typings.dart | agents/src/constants.ts | Transcriptions | LiveKit Docs
l
attributes = { ATTRIBUTE_TRANSCRIPTION_FINAL: "false", # for interim updates ATTRIBUTE_TRANSCRIPTION_TRACK_ID: self.track.sid, } writer = await self.room.local_participant.stream_text( topic=TOPIC_TRANSCRIPTION, sender_identity=self.participant_identity, attributes=attributes, ) # When transcription is complete final_attrs = { ATTRIBUTE_TRANSCRIPTION_FINAL: "true", } await writer.aclose(attributes=final_attrs) how does this work
t
Let me walk you through how this code snippet works, step by step: • Interim Transcription: When a transcription segment starts, 
stream_text
is called with 
ATTRIBUTE_TRANSCRIPTION_FINAL
set to 
"false"
. This sets up a new text stream to keep you updated as the transcription progresses. You can use 
writer.write()
to send these interim transcription chunks over the stream. • Final Transcription: Once the transcription segment is complete—like when the end of speech is detected—`aclose()` is called on the 
TextStreamWriter
. At this point, the attributes update to include 
ATTRIBUTE_TRANSCRIPTION_FINAL
set to 
"true"
, letting everyone know that no more updates will be coming for that segment. • Client Handling: Clients who are listening to the text stream check the 
ATTRIBUTE_TRANSCRIPTION_FINAL
attribute to understand whether the transcription is still being updated or if it’s finalized. This helps make sure the final transcript is shown correctly. This approach helps keep everything in sync smoothly between real-time speech processing and the clients receiving the transcription, giving a clear signal for when each transcription segment is complete. I hope this clarifies how it all fits together! Sources: livekit-plugins/livekit-plugins-fal/livekit/plugins/fal/stt.py | agents/src/transcription.ts | hooks/useDataStreamTranscriptions.ts | Sources/LiveKit/Types/Attributes/AttributeTypings.swift
l
any other way to ensure last user speech text is part of chat context
t
I want to make sure you feel confident that the last user speech text is always included in the chat context. LiveKit takes care of this seamlessly through the 
on_user_turn_completed
lifecycle event: • When the user finishes speaking, the system gently detects the end of their utterance using Voice Activity Detection (VAD) or Speech-to-Text (STT) end-of-speech events. • The final transcript is then gathered and added as a 
user_message
to a temporary, editable copy of the chat context. • This updated chat context is passed along to the 
on_user_turn_completed
callback, making sure the most recent user speech is included before generating a thoughtful response. On top of that, you have the flexibility to use the 
on_user_turn_completed
node to tweak the chat context, perform lookups, or add any extra data before the response is created. This way, you can be confident that the user's latest speech is always part of the conversation. Sources: livekit-plugins/livekit-plugins-ultravox/livekit/plugins/ultravox/realtime/... | agents/src/voice/agent_activity.ts | External data and RAG | LiveKit Docs
l
how to detect if there is an uncommited speech from user
t
Let's explore how you can gently detect if a user has uncommitted speech. Here are some helpful signs to look out for: • Current Transcript: Take a peek at 
currentTranscript
. If it has text, it means speech has been picked up but hasn't been finalized just yet. • Speaking Activity: Keep an eye on VAD events like 
START_OF_SPEECH
and 
END_OF_SPEECH
. When 
speaking
is 
true
, it shows the user is actively speaking, so their speech is still in progress. • Interim and Final Transcripts: Watch for 
INTERIM_TRANSCRIPT
and 
FINAL_TRANSCRIPT
events. Interim transcripts are like little glimpses of speech that’s still being processed. • Turn Commit Status: Check if 
userTurnCommitted
is 
false
. When this flag is 
false
and 
currentTranscript
isn’t empty, it means the speech hasn’t been finalized yet. By keeping these indicators in mind, you’ll be able to gently recognize when speech has been detected but is still waiting to be finalized as a complete user turn. I hope this helps you feel more confident in managing speech detection! Sources: agents/src/voice/agent_activity.ts | AI voice agents | LiveKit Docs
Open in Slack
PreviousNext
Powered by