https://livekit.io logo
Join Slack
Powered by
could sharing the tts between multiple agent be ca...
# ask-ai
c
could sharing the tts between multiple agent be causing this error making the agent speech jerky/jittery? Starting with this warning: 
WARNING livekit.agents - _SegmentSynchronizerImpl text marked as ended in capture text, rotating segment
Cascading to these errors: 
2025-07-22 17:24:54,862 - ERROR livekit.agents - Error in _audio_forwarding_task
Traceback (most recent call last):
File "/Users/clovishublin/Desktop/mirror-repos/livekit-agent-service/venv/lib/python3.9/site-packages/livekit/agents/utils/log.py", line 16, in
async_fn_logs
return await fn(*args, **kwargs)
File "/Users/clovishublin/Desktop/mirror-repos/livekit-agent-service/venv/lib/python3.9/site-packages/livekit/agents/voice/generation.py", line
302, in _audio_forwarding_task
await audio_output.capture_frame(frame)
File "/Users/clovishublin/Desktop/mirror-repos/livekit-agent-service/venv/lib/python3.9/site-packages/livekit/agents/voice/transcription/synchroni
zer.py", line 498, in capture_frame
await self._next_in_chain.capture_frame(frame)  # passthrough audio
File "/Users/clovishublin/Desktop/mirror-repos/livekit-agent-service/venv/lib/python3.9/site-packages/livekit/agents/voice/room_io/_output.py",
line 83, in capture_frame
await self._audio_source.capture_frame(frame)
File "/Users/clovishublin/Desktop/mirror-repos/livekit-agent-service/venv/lib/python3.9/site-packages/livekit/rtc/audio_source.py", line 149, in
capture_frame
raise Exception(cb.capture_audio_frame.error)
Exception: an RtcError occured: InvalidState - failed to capture frame
2025-07-22 17:24:54,935 - WARNING livekit.agents - _SegmentSynchronizerImpl audio marked as ended in capture audio, rotating segment
2025-07-22 17:24:54,937 - ERROR livekit.agents - capture_frame called while flush is in progress
t
Thanks for reaching out, Clovis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Sharing a TTS engine or resource between multiple agents is not explicitly documented as a cause of jerky or jittery agent speech, nor is it directly linked to segment rotation warnings in LiveKit's agent service logs. The error 
InvalidState - failed to capture frame
typically occurs when the 
rtc.AudioSource
queue is overloaded. To address this, increase the 
queue_size_ms
parameter when creating your 
AudioSource
instance. For example:
Copy code
audio_source = rtc.AudioSource(
    sample_rate=sample_rate,
    num_channels=channels,
    queue_size_ms=2000  # Increase from default to prevent frame capture failures
)
Larger 
queue_size_ms
values (such as 1000-2000ms) provide more buffer room and help prevent frame capture failures. LiveKit's TTS architecture is designed for each agent to maintain its own TTS processing pipeline. Sharing TTS resources across multiple agents is not a standard pattern and could introduce conflicts or resource contention, but this is not directly cited as a cause of the errors or warnings you've described. To resolve the jerky speech and related errors, focus on increasing the audio buffer size as shown above. If you are sharing TTS resources between agents, consider isolating TTS pipelines per agent to avoid potential conflicts. Sources: Speech & Audio Processing | livekit/livekit_composite | DeepWiki | Resolving "InvalidState - failed to capture frame" errors in AudioSource | livekit-agents/livekit/agents/voice/agent_session.py
c
okay but how do I set this up in the session.start()? what keyword to use? audio_source?
t
Hey there! Let me help you understand how to set the 
queue_size_ms
parameter. It's actually configured right when you create your 
rtc.AudioSource
instance, not through 
session.start()
. Think of it like setting up your audio buffer before hitting the road. Here's a quick example to show you:
Copy code
audio_source = rtc.AudioSource(
    sample_rate=sample_rate,
    num_channels=channels,
    queue_size_ms=2000  # Set buffer size here
)
Just a heads up ā€“ there's no 
queue_size_ms
or 
audio_source
option in 
session.start()
. Make sure you set up your 
AudioSource
with the buffer size you want before kicking off your session. Pro tip: We usually recommend setting 
queue_size_ms
between 1000-2000ms. This helps keep your audio streaming smooth and prevents those annoying frame capture hiccups. šŸ˜Š Sources: livekit-rtc/livekit/rtc/audio_source.py | livekit-agents/livekit/agents/voice/agent_session.py | Resolving "InvalidState - failed to capture frame" errors in AudioSource
c
like this? 
audio_source = AudioSource(
sample_rate=24000,  # Match TTS provider sample rate
num_channels=1,
queue_size_ms=2000  # Increase from default ~500ms to prevent buffer overrun
)
<http://logger.info|logger.info>(f"[{session_id}] :loud_sound: Configured AudioSource with 2000ms queue buffer to prevent frame capture failures")
try:
await session.start(
room=ctx.room,
agent=agent,
audio_source=audio_source,  # Add the configured AudioSource
room_input_options=room_input_options,
room_output_options=RoomOutputOptions(
sync_transcription=True,
),
t
I don't have the answer you're looking for. You could also try asking your question: ā€¢ in one of the other Slack channels or ā€¢ to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
c
are you sure about what you said me, I am getting this error copying your stuff: ERROR livekit-agent - Session error for agent 4b0dd690-f6fd-4746-83f0-ccaee70b624d, session session_1753239128140_npi34tu: start() got an unexpected keyword argument 'audio_source'
2 Views
Open in Slack
PreviousNext
Powered by