Hello, in our company we are using livekit for our intercom app. For every call there are 2 participant, the caller and the receiver. I'm running the python agent, but seems that only the caller text is being transcript, I only see his text in the log. The relevant part of the agent is:

session = AgentSession(
        stt= azure.STT(
            language=["it-IT", "en-US"],
        ),

        turn_detection=MultilingualModel(),
        vad=ctx.proc.userdata["vad"],

        use_tts_aligned_transcript=True,
    )

...

await session.start(
        agent=Assistant(),
        room=ctx.room,
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVC(),
        ),
    )

My expectation is to have the agent listen for both the participant. Am I doing something wrong?

Thanks for reaching out, Piero. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

the library is shared the call works correctly

To make sure you get transcription for both participants in your LiveKit call, it’s important to manage audio tracks for each participant as they join. A good way to do this is by subscribing to the

track_subscribed

event and starting a transcription session for each participant’s audio track. Here’s a simple approach to guide you: 1. Start by initializing the agent session with your STT (like Azure) and VAD (Voice Activity Detection) models. 2. Listen for the

track_subscribed

event to know when a participant’s audio track is available. 3. Create an audio stream and an STT stream for each participant’s audio track. 4. Feed audio frames into the STT stream to get real-time transcription going. If you’d like, I’m here to help with a detailed example or any other questions you have. Just let me know! Sources: server/main.py | livekit-agents/livekit/agents/voice/room_io/room_io.py