delightful-student-35473
02/28/2025, 12:24 PMgray-monkey-83525
02/28/2025, 12:24 PMdelightful-student-35473
02/28/2025, 12:30 PMgray-monkey-83525
02/28/2025, 12:30 PMdelightful-student-35473
02/28/2025, 12:45 PMasync def _eg_single_segment(
text: str,
tts_forwarder: transcription.TTSSegmentsForwarder,
tts_aws: tts.TTS,
source: rtc.AudioSource,
room: rtc.Room,
):
"""Transcription example without streaming (single string)"""
<http://logger.info|logger.info>("pushing text %s", text)
tts_forwarder.push_text(text)
tts_forwarder.mark_text_segment_end()
playout_q = asyncio.Queue[Optional[rtc.AudioFrame]]()
playout_task = asyncio.create_task(_playout_task(tts_forwarder, playout_q, source))
async for output in tts_aws.synthesize(text):
tts_forwarder.push_audio(output.frame)
playout_q.put_nowait(output.frame)
tts_forwarder.mark_audio_segment_end()
playout_q.put_nowait(None)
await playout_task
if i am pushing multiple audio with tts forwarder in the same time can i filter the audio in the frontend based on th elanguage?gray-monkey-83525
02/28/2025, 12:45 PMTTSSegmentsForwarder
class allows you to specify a language for each transcription segment. This language information can be used to filter audio on the frontend.
To implement this, you would need to ensure that each audio segment pushed to the TTSSegmentsForwarder
includes the appropriate language metadata. On the frontend, you can then use this metadata to selectively play or display audio segments based on the desired language.
Would you like more detailed guidance on how to implement this filtering mechanism?
Sources:
- LiveKit documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.delightful-student-35473
02/28/2025, 12:46 PMgray-monkey-83525
02/28/2025, 12:46 PMdelightful-student-35473
02/28/2025, 12:49 PMgray-monkey-83525
02/28/2025, 12:49 PMdelightful-student-35473
02/28/2025, 12:50 PM# Prepare a forwarder for TTS events
tts_forwarder = transcription.TTSSegmentsForwarder(
room=ctx.room,
participant=ctx.room.local_participant,
language=lang_code,
)
gray-monkey-83525
02/28/2025, 12:50 PMTTSSegmentsForwarder
, make sure the language
parameter is set correctly for each segment, as shown in your code snippet. This metadata will be crucial for filtering on the frontend.
2. Transmit Language Metadata to Frontend: Ensure that the language metadata is transmitted alongside the audio data to the frontend. This can be done by including the language information in the events or data structures that carry the audio frames.
3. React Component for Audio Playback: In your React application, create a component that listens for audio events and filters them based on the language. You can use state to manage the current language filter and conditionally render audio elements.
4. Implement Filtering Logic: Use the language metadata to filter audio segments. For example, you can maintain a list of audio segments and their associated languages, and only render or play those that match the selected language.
Here's a basic example of how you might implement this in React:
import React, { useState, useEffect } from 'react';
function AudioPlayer({ audioSegments, selectedLanguage }) {
const [filteredSegments, setFilteredSegments] = useState([]);
useEffect(() => {
// Filter audio segments based on the selected language
const filtered = audioSegments.filter(segment => segment.language === selectedLanguage);
setFilteredSegments(filtered);
}, [audioSegments, selectedLanguage]);
return (
<div>
{filteredSegments.map((segment, index) => (
<audio key={index} controls>
<source src={segment.audioUrl} type="audio/wav" />
Your browser does not support the audio element.
</audio>
))}
</div>
);
}
export default AudioPlayer;
In this example, audioSegments
is an array of objects where each object contains an audioUrl
and language
. The selectedLanguage
is the language you want to filter by. The component filters the segments and renders only those that match the selected language.
Would you like more details on any specific part of this implementation?
Sources:
- https://docs.livekit.io/agents/voice-agent/transcriptions/
- https://docs.livekit.io/python/livekit/agents/transcription/tts_forwarder.html#args 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.delightful-student-35473
03/03/2025, 7:53 AMrack = rtc.LocalAudioTrack.create_audio_track("agent-mic", source)
options = rtc.TrackPublishOptions(source=rtc.TrackSource.SOURCE_MICROPHONE)
What this mean? if i changedthe neame should be tracked differently from the frontend?gray-monkey-83525
03/03/2025, 7:53 AM