https://livekit.io logo
Join Slack
Powered by
is it possible to find out which participant is so...
# ask-ai
t
is it possible to find out which participant is source of specific SpeechEvent?
t
Thanks for reaching out, sergey. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
It is not possible to directly determine the participant source from the 
SpeechEvent
object itself, as it does not include participant identification fields. However, you can associate speech events with participants using these approaches: • Transcription Events: Higher-level events like 
UserInputTranscribedEvent
include a 
speaker_id
field that identifies the participant. • Agent or Session Context: When processing speech events in an agent, you can maintain a mapping between the participant and the events by tracking the participant identity in your session or agent logic. • Track-Based Mapping: Use utility functions to map audio tracks to participant identities, then associate incoming speech events with the correct participant based on the track. • Fixed Participant Processing: Configure your agent or session to process audio from a specific participant, ensuring all speech events are attributed to that participant. Participant identification for speech events is typically managed at the agent or session level, not within the 
SpeechEvent
object itself. You will need to maintain this association in your application logic. Sources: SpeechEvent | LiveKit Agents | livekit-agents/livekit/agents/voice/events.py | agents/src/multimodal/multimodal_agent.ts | How to set a fixed participant for agent audio processing
t
why 
speaker_id
can be None in 
UserInputTranscribedEvent
?
t
Hey there! Let me break down why the 
speaker_id
field in 
UserInputTranscribedEvent
might sometimes show up as 
None
. It's all about how your Speech-to-Text (STT) plugin handles speaker tracking. • STT Plugin Support: Some speech-to-text services just aren't set up to tell different speakers apart. In those cases, 
speaker_id
will always be 
None
. No worries! • Diarization Not Enabled: Even if your STT plugin can technically do speaker identification (like Speechmatics), you've got to turn that feature on. Think of it like a light switch - if it's not flipped on (like setting 
diarization="speaker"
), 
speaker_id
stays 
None
. • Single Speaker or Not Needed: Sometimes you're just talking solo, or speaker details don't matter for what you're doing. In those situations, 
speaker_id
will also be 
None
. The cool thing is, LiveKit is super flexible. We've designed this system to work smoothly with different speech-to-text providers, making sure you're covered no matter what your specific setup looks like! Sources: livekit-agents/livekit/agents/voice/agent_activity.py | Speechmatics STT integration guide | LiveKit Docs | Events and error handling | LiveKit Docs
t
how to enable diarization?
t
Hey there! Want to make your LiveKit conversations more personalized? Speaker diarization is your new best friend. It helps you track who's saying what, making your transcriptions super clear. Let me walk you through how to set it up with the Speechmatics STT plugin. 1. First, initialize your STT with 
diarization="speaker"
and tweak those speaker settings:
Copy code
from livekit.plugins import speechmatics

stt = speechmatics.STT(
    transcription_config=speechmatics.types.TranscriptionConfig(
        language="en",
        operating_point="enhanced",
        enable_partials=True,
        max_delay=0.7,
        diarization="speaker",
        speaker_diarization_config={"max_speakers": 2},  # Adjust as needed
    )
)
• max_speakers: How many voices do you want to track? You can choose between 2 and 100! • speaker_sensitivity: Fine-tune how precisely you want to detect different speakers • prefer_current_speaker: Helps prevent unnecessary speaker switches The cool part? When diarization is on, you'll get a handy 
speaker_id
with each 
UserInputTranscribedEvent
. This means you can easily tell who's speaking in your transcripts. Pretty neat, right? Sources: Speechmatics STT integration guide | LiveKit Docs | pipeline-stt/diarization.py | Speechmatics integration guide | LiveKit Docs
t
what exactly stt plugins supports diarization?
t
Hey there! Let me break down the speaker diarization support for you in a super friendly way. • Amazon Transcribe • Speechmatics If you're using Speechmatics, you're in luck! You can easily turn on diarization with the 
diarization="speaker"
setting. Plus, you've got some cool options to play with like 
max_speakers
, 
speaker_sensitivity
, and 
prefer_current_speaker
. Pretty neat, right? Just a heads up, these STT plugins don't currently support speaker diarization: • AssemblyAI • Azure AI Speech • Azure OpenAI • OpenAI • Sarvam • Spitch Oh, and there's a quick mention of Deepgram having a 
diarize
option, but it's not officially in the main lineup. Just thought you'd want to know! Sources: Speech-to-text (STT) integrations | LiveKit Docs | Speechmatics STT integration guide | LiveKit Docs | STTOptions | LiveKit Agents
t
is it possible to do stt only for specific participant?
t
Great news! With LiveKit, you can absolutely do speech-to-text transcription for specific participants. Let me break down how you can make this work smoothly. • Fixed Participant: Want to focus on just one person? No problem! Just set the 
participant
parameter in the 
RoomIO
constructor to the specific participant's identity. This way, your agent will only process audio from that particular person.
Copy code
session = AgentSession(llm=openai.realtime.RealtimeModel())
room_io = RoomIO(session, room=ctx.room, participant="participant-identity")
await room_io.start()
await session.start(agent=MyAgent())
• Multiple Specific Participants: Need to track more than one person? You can totally create separate agent sessions for each participant you want to transcribe. • Track-Specific: If you want even more granular control, you can transcribe specific audio tracks directly from specific participants. • Dynamic Switching: Want to change things up on the fly? Use 
set_participant()
on 
RoomIO
to switch which participant's audio is being transcribed in real-time. These flexible options give you complete control over participant transcription in your application. Pretty cool, right? Sources: livekit-agents/livekit/agents/voice/room_io/room_io.py | Transcriptions | LiveKit Docs | Speech & Audio Processing | livekit/livekit_composite | DeepWiki | How to set a fixed participant for agent audio processing
t
is it possible to start multiple AgentSessions from the same entrypoint?
t
Absolutely! Starting multiple 
AgentSession
instances in LiveKit is totally doable, and I'll walk you through a friendly approach. Here's a neat way to handle this: you can create a new 
AgentSession
for each participant or scenario, and then kick them off as separate async tasks using 
asyncio.create_task()
. Let me show you an example:
Copy code
async def entrypoint(ctx: JobContext):
    await ctx.connect(auto_subscribe=agents.AutoSubscribe.AUDIO_ONLY)

    @ctx.room.on("participant_connected")
    def on_participant_connected(participant: rtc.RemoteParticipant):
        session = AgentSession(...)
        asyncio.create_task(
            session.start(
                room=ctx.room,
                agent=NavigatorAgent()
            )
        )
• Feel free to create as many 
AgentSession
instances as you need • Start each session concurrently with 
asyncio.create_task()
- it's super flexible! • You can share the same 
JobContext
and room, but customize configurations or agents for each session This approach gives you the power to handle multiple participants or scenarios smoothly and efficiently, all from a single entrypoint. Pretty cool, right? Sources: complex-agents/ivr-agent/agent.py
4 Views
Open in Slack
PreviousNext
Powered by