https://livekit.io logo
Join Slack
Powered by
I'm making an agent system using Unity. To get tr...
# ask-ai
b
I'm making an agent system using Unity. To get transcriptions I call Room.RegisterTextStreamHandler("lk.transcription", OnTranscriptionStream), where OnTranscriptionStream is the callback. For my agent I get correct transriptions. But when the user speaks I get a transcription but it is always empty. How should I be getting transcriptions for user speech?
t
Thanks for reaching out, Mat. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — that’s a clear description. You’re already using the correct handler (
Room.RegisterTextStreamHandler("lk.transcription", ...)
) for transcriptions; the issue is how the transcription stream messages are being read and filtered. What to check and change • Only act on streams that are transcriptions of audio: the stream will include the attribute 
lk.transcribed_track_id
. If that attribute is missing, the message is a text message (not STT). • Only process final transcripts: the attribute 
lk.transcription_final
will be 
"true"
for the final completed transcription. Interim updates can be empty or partial. • Read the stream payload fully with 
ReadAll()
(or your SDK equivalent) and assemble the returned messages into the full transcription before using it. Suggested Unity (C#) pattern 1. Register the handler on 
lk.transcription
. 2. In the callback: ◦ Check 
reader.Info.Attributes
for 
lk.transcribed_track_id
(confirm audio STT). ◦ Check 
lk.transcription_final == "true"
(confirm final transcript). ◦ Call 
await reader.ReadAll()
and combine the returned pieces into a single string. ◦ Only then process/display the transcription. Short example outline:
Copy code
room.RegisterTextStreamHandler("lk.transcription", async (reader, participantIdentity) =>
{
    var attrs = reader.Info.Attributes;
    var isTranscription = attrs.ContainsKey("lk.transcribed_track_id");
    var isFinal = attrs.TryGetValue("lk.transcription_final", out var v) && v == "true";

    if (!isTranscription) return; // not audio STT
    if (!isFinal) return; // skip interim or handle separately

    var pieces = await reader.ReadAll();
    var full = string.Concat(pieces);
    if (!string.IsNullOrEmpty(full))
    {
        Debug.Log($"Final transcription from {participantIdentity}: {full}");
        // handle transcription...
    }
});
Why you were getting empty transcriptions • If you read interim streams (or read before the final segment arrives) you may receive empty/partial results. • If you didn’t check for 
lk.transcribed_track_id
, you might be treating non-audio text messages as STT and getting empty content. If you want, tell me: • Whether you currently check 
lk.transcription_final
and 
lk.transcribed_track_id
in your handler. • If you still see empty final transcripts after implementing the checks above (and I’ll suggest next diagnostics). Sources: Text and transcriptions | LiveKit Documentation
b
I am checking all of the values you mention. But all of the transcriptions I get for the user are empty strings.
t
Thanks — I appreciate you confirming that you’re already checking the attributes. Since you still get empty strings for user speech, I need a few diagnostic details so I can pinpoint the cause and suggest the next steps. Please share the following: 1) A short excerpt of the handler code you’re using (the callback you pass to 
RegisterTextStreamHandler
) — specifically how you: • Inspect 
reader.Info.Attributes
• Call 
ReadAll()
(or equivalent) and assemble the pieces • Log or return the final string 2) Are you seeing any values for these attributes on user speech messages? Please paste an example of 
reader.Info.Attributes
for a user transcription message (masked any PII). I’m especially looking for 
lk.transcribed_track_id
, 
lk.transcription_final
, and 
lk.segment_id
. 3) Are users sending audio from a local microphone track in the same client, or are they remote participants? In other words, is the transcribed track coming from a published audio track in the Room? 4) How are you publishing the user’s microphone audio? Briefly: • Which Unity SDK version are you using? • Which method do you use to publish audio (e.g., 
LocalAudioTrack.CreateFromDevice()
or custom capture)? • Any custom audio processing (resampling, muting, track replacement) before publishing? 5) Any logs or console errors when the user speaks (SDK warnings about audio, track state, or transcription)? If possible, paste the relevant log lines. 6) Are agent transcriptions always non-empty while user transcriptions are empty? If yes, confirm whether the agent’s audio is published differently (e.g., generated TTS or a text stream) compared to the user microphone track. What I’ll do with this info • If the attributes are present but payloads are empty, I’ll suggest targeted checks (track publication state, STT configuration, segment timing). • If 
lk.transcribed_track_id
is missing for user messages, I’ll explain why those messages are not STT and how to ensure the audio track is eligible for transcription. • If you provide code I’ll point out any likely issues (e.g., reading the stream too early, joining pieces incorrectly, or accidentally discarding data). If you prefer, paste only the handler code and one example attribute set + one console log sample and I’ll start from there.
Open in Slack
PreviousNext
Powered by