important-kitchen-1796
11/19/2025, 9:49 AMTTS metrics {"ttfb": ..., "audio_duration": ...}
...
TTS metrics {"ttfb": ..., "audio_duration": 21.25}
• Then we see:
INFO livekit.agents - playback finished event received {"caller_identity": "bey-avatar-room-fdfd6b2c"}
INFO __mp_main__ - Audio sink attached to session: _SyncedAudioOutput(... DataStreamAudioOutput(label='DataStreamIO', next=None))
• User replies, more LLM + TTS metrics, and again:
playback finished event received {"caller_identity": "bey-avatar-room-fdfd6b2c"}
So in Session-1:
• session.output.audio is correct: TranscriptSynchronizer → DataStreamAudioOutput.
• The Bey avatar plays the audio and sends back playback_finished twice.
• Whole loop is healthy.
❌ Session-2 (this is the broken one)
You’ve now got much richer logs here, and they’re very telling:
1. Avatar setup:
DEBUG livekit.plugins.bey - starting avatar session
...
DEBUG livekit.agents - waiting for the remote participant {"identity": "bey-avatar-room-076d289b"}
DEBUG livekit.agents - waiting for the remote track {"identity": "bey-avatar-room-076d289b", "kind": "KIND_VIDEO"}
...
DEBUG livekit.agents - remote participant ready {"identity": "bey-avatar-room-076d289b"}
DEBUG livekit.agents - using audio io: `RoomIO` -> `AgentSession` -> `TranscriptSynchronizer` -> `DataStreamIO`
DEBUG livekit.agents - using transcript io: `AgentSession` -> `TranscriptSynchronizer` -> `RoomIO`
This tells us:
• The avatar participant connects correctly for this room.
• DataStreamAudioOutput is set up as expected.
1. Audio sink attached:
INFO __mp_main__ - Audio sink attached to session: _SyncedAudioOutput(label='TranscriptSynchronizer', next=DataStreamAudioOutput(label='DataStreamIO', next=None))
So Session-2 has the exact same audio sink shape as Session-1. Good.
1. TTS & LLM are clearly working multiple times:
We see:
LLM metrics {... completion_tokens": 65 ...}
TTS metrics {"ttfb": 1.234..., "audio_duration": 3.73}
TTS metrics {"ttfb": 1.27..., "audio_duration": 3.15}
...
TTS metrics {"ttfb": 1.07..., "audio_duration": 3.71}
LLM metrics {"completion_tokens": 103, ...}
TTS metrics {"ttfb": 1.63..., "audio_duration": 2.51}
TTS metrics {"ttfb": 2.41..., "audio_duration": 21.2}
...
TTS metrics {"ttfb": 1.74..., "audio_duration": 3.61}
LLM metrics {"completion_tokens": 102, ...}
TTS metrics {"ttfb": 0.85..., "audio_duration": 2.36}
TTS metrics {"ttfb": 2.10..., "audio_duration": 21.03}
So in Session-2:
• Multiple turns of:
◦ Your tool is called and returns questions.
◦ LLM responds.
◦ OpenAI TTS produces audio (non-zero duration).
• The agent is not dead; it keeps generating answers and audio.
1. Critical weird event:
Right after the first conversation_item_added role=user in session-2:
INFO livekit.agents - LLM metrics {"prompt_tokens": 0, "completion_tokens": 0}
INFO livekit.agents - TTS metrics {"ttfb": -1.0, "audio_duration": 0.0}
This indicates:
• There was a “no-op” or aborted segment:
◦ No LLM tokens generated,
◦ TTS thinks “no audio” (ttfb=-1, duration=0).
• This usually happens when a generation is cancelled or something internal decides "nothing to say".
Not fatal by itself, but it shows there is some cancellation logic kicking in.
1. The biggest missing piece:
Despite all those TTS metrics, there is never a single:
playback finished event received {"caller_identity": "bey-avatar-room-076d289b"}
in Session-2.
And you also don’t see our custom handler log ([audio_sink playback_finished]), which would fire on that event.
So, in Session-2:
• Avatar session is created.
• Remote participant is ready.
• Audio sink is correctly attached (TranscriptSynchronizer → DataStreamAudioOutput).
• LLM + TTS produce audio for several turns.
• But the avatar never once sends playback_finished back.
That’s the core bug.
2. Do I see an issue in your code?
Short answer: no, your wiring looks correct now.
• avatar_participant_identity=f"bey-avatar-{ctx.room.name}" → good; avoids identity collisions.
• avatar.start(session, room=ctx.room) before session.start() → matches Bey docs.
• Audio sink debug shows DataStreamAudioOutput is in place in both sessions.
• The only real anomaly in metrics is that one (prompt_tokens=0, completion_tokens=0, ttfb=-1.0) event, which is strongly suggestive of an internal cancel / “false interruption” (and you have that warning about resume_false_interruption too).
Nothing there looks like “you mis-used the API”.
From your logs, this looks like:
For the second session in the same Python process, the avatar’s DataStream pipeline never acknowledges any of the audio segments, even though they are generated and sent.Which is exactly why you: • Hear/see the first question (initial segment might be special-case), • And then behaviour gets weird. 3. Why this explains the behaviour you see Your mental model:
Session-1:
TTS metrics ✅
✅playback finished
→ Agent keeps flowing.
Session-2:
TTS metrics ✅
❌playback finished
→ Agent “stops responding”.This is exactly what the logs confirm: • From your side: everything up to sending audio is working. • From the avatar side: ◦ Session-1: it plays and acknowledges. ◦ Session-2: it never acknowledges any playback. So the problem is no longer “is my Python code wrong?” It’s “why does the avatar/DataStream path stop sending
playback_finished on the second job in the same worker?”.
That’s almost certainly a bug/edge case in:
• livekit.plugins.bey + DataStreamAudioOutput interaction, or
• The remote Bey avatar worker for subsequent connections.
4. What I’d do next (concrete steps)
Since you’ve already done the heavy debugging, you’re at the point where:
1. Your own workarounds / tests:
◦ Keep using no avatar for now in production (since that path is 100% stable for you).
• If you want to still use the avatar in demos:
• Run one session per worker process, i.e. restart the Python worker for each candidate interview.
◦ This is ugly but will likely “hide” the bug in practice since every session then behaves like Session-1.
2. For LiveKit / Bey support (you now have a great story):
3. You can literally say:
◦ “Here is Session-1 log: TTS metrics + playback_finished.”
• “Here is Session-2 log: TTS metrics but no playback_finished ever, despite:
▪︎ avatar remote participant ready,
▪︎ audio sink correctly attached,
◦ and multiple TTS segments with non-zero duration.”
◦ “Same code, same worker process, only difference is the job/room.”
4. That’s a very clear, minimal repro for them.
If you’d like, I can now:
• Draft a clean email / GitHub issue description using exactly these logs, with bullet points and the minimal code snippet, so you can just copy-paste it to the LiveKit or Bey team.lively-winter-88668
11/19/2025, 11:29 AM