chilly-spring-61928
12/16/2025, 4:51 PMasync def transcription_node(
self, text: AsyncIterable[str | TimedString], model_settings: ModelSettings
) -> AsyncGenerator[str | TimedString, None]:
"""
Intercepts agent transcription to stream chunks via Redis.
IMPORTANT: To avoid orphaned streams when user interrupts before audio plays,
we only start streaming once audio has actually started (agent_state == "speaking").
This follows the same pattern as LiveKit's TranscriptSynchronizer which gates
text output based on audio timing.
The final message is handled separately by conversation_item_added event,
which calls session_recorder.log_and_stream_conversation_item.
Message ID continuity is maintained via transcription_drafts.
"""
session_recorder = getattr(self.session.userdata, "session_recorder", None)
# Track streaming state
stream_started = False
buffered_chunks: list[str | TimedString] = []
# Small polling interval to check for audio start
AUDIO_CHECK_INTERVAL = 0.05 # 50ms
async def wait_for_audio_or_interruption() -> bool:
"""
Wait until audio starts playing (agent_state == 'speaking') or speech is interrupted.
Returns True if audio started, False if interrupted before audio.
"""
while True:
# Check if audio has started
if self.session.agent_state == "speaking":
return True
# Check if speech was interrupted before audio started
speech_handle: SpeechHandle | None = getattr(self.session, "current_speech", None)
if speech_handle and speech_handle.interrupted:
<http://logger.info|logger.info>("[TRANSCRIPTION] Speech interrupted before audio started - discarding buffer")
return False
# Brief wait before checking again
await asyncio.sleep(AUDIO_CHECK_INTERVAL)
async def start_stream_and_flush_buffer():
"""Start the transcription stream and flush all buffered chunks."""
nonlocal stream_started
if stream_started or not session_recorder:
return
stream_started = True
try:
await session_recorder.start_agent_transcription_stream()
<http://logger.info|logger.info>(
"[TRANSCRIPTION] Audio started - flushing %d buffered chunks",
len(buffered_chunks)
)
# Flush all buffered chunks
for buffered_delta in buffered_chunks:
if buffered_delta:
await session_recorder.publish_agent_transcription_chunk(buffered_delta)
except Exception as e:
logger.warning("Failed to start transcription stream: %s", e)
async for delta in text:
# Check current speech handle for interruption
speech_handle: SpeechHandle | None = getattr(self.session, "current_speech", None)
if not stream_started:
# Audio hasn't started yet - buffer the chunk
if delta:
buffered_chunks.append(delta)
# Check if audio has started (agent is now speaking)
if self.session.agent_state == "speaking":
await start_stream_and_flush_buffer()
elif speech_handle and speech_handle.interrupted:
# Interrupted before audio started - discard buffer and stop
<http://logger.info|logger.info>(
"[TRANSCRIPTION] Interrupted before audio - discarding %d chunks",
len(buffered_chunks)
)
buffered_chunks.clear()
# Still yield the delta to maintain the pipeline
yield delta
continue
else:
# Stream is active - publish chunk directly
if session_recorder and delta:
try:
await session_recorder.publish_agent_transcription_chunk(delta)
except Exception as e:
logger.warning("Failed to publish transcription chunk: %s", e)
yield delta
# After all chunks processed, check if we need to start stream
# (handles edge case where audio starts after last chunk arrives)
if not stream_started and buffered_chunks:
# Give a brief moment for audio to start
audio_started = await asyncio.wait_for(
wait_for_audio_or_interruption(),
timeout=0.5 # Max 500ms wait
) if buffered_chunks else False
if audio_started:
await start_stream_and_flush_buffer()
else:
<http://logger.info|logger.info>(
"[TRANSCRIPTION] Stream ended without audio start - "
"discarding %d buffered chunks (no orphan created)",
len(buffered_chunks)
)tall-airline-16756
12/16/2025, 4:51 PMtall-airline-16756
12/16/2025, 4:52 PMagent_state on a timer. That simplifies the code, removes CPU/timing overhead, and eliminates certain race windows.
Concise suggestions:
• Use SpeechHandle lifecycle callbacks (e.g., on_start / on_interrupted) to:
◦ start the transcription stream and flush buffered chunks when audio actually starts
◦ clear the buffer when interrupted before start
• If SpeechHandle events aren’t available in your SDK version, create a single small task that awaits the speech handle’s events (or state change) once per speech session — don’t poll inside the per-chunk loop.
• Keep your final fallback (short wait after the last chunk) only if you expect audio to start slightly after the last text chunk; otherwise it can be removed when using event-driven handlers.
• Continue to yield deltas so the rest of the pipeline stays intact; publishing to session_recorder can remain async (wrapped with try/except as you have).
If you’d like, I can:
• show a concise event-driven code snippet adapted to your snippet, or
• sketch the small monitoring-task approach if SpeechHandle lacks event hooks. Which would you prefer? Sources: Agent speech and audio | LiveKit docs | Pipeline nodes and hooks | LiveKit docs | Agents (Python) | livekit/livekit_composite | DeepWiki | Observability data hooks | LiveKit docs | Text and transcriptions | LiveKit docschilly-spring-61928
12/16/2025, 4:55 PMtall-airline-16756
12/16/2025, 4:56 PMSpeechHandle in the SDK sources I have.
Given that, your current approach (buffering chunks and gating publish on agent_state / current_speech.interrupted) is appropriate. Two concise options moving forward:
• Keep your buffered/polling approach but reduce polling surface:
◦ Poll only once per speech session (a short watcher task started when current_speech is created) rather than polling inside the per-chunk loop.
◦ This watcher awaits either audio-start (check agent_state == "speaking") or interruption, then flushes or discards the buffer.
• Use AgentSession events instead of SpeechHandle events:
◦ Listen for agent_state_changed (or other session-level events available in your SDK) to detect when the agent moves into a speaking state and trigger flush/start there.
If you tell me which SDK (Python, Node, Flutter, etc.) you’re using, I’ll provide a compact snippet for the watcher-task approach or an example using agent_state_changed. Which would you prefer?
Sources: client-sdk-flutter | livekit/livekit_composite | DeepWiki | Text and transcriptions | LiveKit docs | Agents and handoffs | LiveKit docs | Swift quickstart | LiveKit docs | Pipeline nodes and hooks | LiveKit docs