https://livekit.io logo
Join Slack
Powered by
How do you structure a conversation with a user an...
# ask-ai
c
How do you structure a conversation with a user and two agentsessions liked respectively to an avatarsession?
t
Thanks for reaching out, Bryce. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To structure a conversation with a user and two agent sessions linked to avatar sessions in LiveKit, follow these steps: 1. Create Agent Sessions: Initialize two 
AgentSession
instances for each agent. 2. Initialize Avatar Sessions: Choose an avatar provider (e.g., Tavus, Beyond Presence) and create their respective 
AvatarSession
objects with the necessary configurations. 3. Start Avatar Sessions: Start each avatar session and join them to the LiveKit room associated with their respective agent sessions. 4. Start Agent Sessions: Each agent joins the room and listens for inputs while their respective avatar handles the visual and audio output. Here is an example setup:
Copy code
async def entrypoint(ctx: JobContext):
    # Create first agent session
    session_1 = AgentSession(
        llm=openai.realtime.RealtimeModel(voice="alloy"),
    )

    # Initialize Tavus avatar
    tavus_avatar = tavus.AvatarSession(
        replica_id="your-tavus-replica-id",
        persona_id="your-tavus-persona-id",
    )
    await tavus_avatar.start(session_1, room=ctx.room)

    # Start first agent
    await session_1.start(
        agent=Agent(instructions="Hello, I'm the first agent!"),
        room=ctx.room,
    )

    # Create second agent session
    session_2 = AgentSession(
        llm=openai.realtime.RealtimeModel(voice="alloy"),
    )

    # Initialize Bey avatar
    bey_avatar = bey.AvatarSession(
        avatar_id="your-bey-avatar-id"
    )
    await bey_avatar.start(session_2, room=ctx.room)

    # Start second agent
    await session_2.start(
        agent=Agent(instructions="Hello, I'm the second agent!"),
        room=ctx.room,
    )
Ensure that each avatar is properly configured with the required IDs and that the necessary environment variables (e.g., API keys) are set up. Let me know if you need further details or assistance with a specific avatar provider.
c
If I do this, the agents talk over each other.
t
To help prevent agents from talking over each other in LiveKit, here are some friendly tips you can try out: 1. Assign Fixed Participants: You can use the 
RoomIO
constructor to set a 
participant
parameter for each agent session. This way, each agent listens to audio from just one participant, keeping things neat and clear.
Copy code
session = AgentSession(...)
room_io = RoomIO(session, room=ctx.room, participant="participant-identity")
await room_io.start()
await session.start(agent=MyAgent())
2. Control Interruptions: Try setting 
allow_interruptions
to false to make sure agents don’t interrupt each other while speaking.
Copy code
session = AgentSession(
    allow_interruptions=False,
    discard_audio_if_uninterruptible=True
)
3. Set Interruption Timeouts: You can also adjust 
agent_false_interruption_timeout
to control how long the agent waits before marking an interruption as false. This helps keep conversations smooth.
Copy code
session = AgentSession(
    agent_false_interruption_timeout=4.0
)
4. Separate Sessions for Participants: Creating separate agent sessions for each participant is another great way to keep audio processing organized and avoid overlap.
Copy code
room_io = RoomIO(
    session=session,
    room=ctx.room,
    participant=participant.identity
)
Putting these strategies together should make managing multiple agent sessions a lot easier and keep everyone from talking over each other. If you want me to walk you through any part or need a hand with the setup, just let me know—I’m here to help! Sources: How to set a fixed participant for agent audio processing | livekit-agents/livekit/agents/voice/agent_session.py
Open in Slack
PreviousNext
Powered by