i am using 'audioMixing: AudioMixing.DUAL_CHANNEL_AGENT' but left channel is empty. i am using beyond presence avatar for agent

Thanks for reaching out, s. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

avatar is also speaking on the right channel, is it possible that avatar counted as human instead of agent?

Thanks — I’m responding to your question about whether the avatar might be counted as a human instead of the agent, and why the left channel is empty when using

audioMixing: AudioMixing.DUAL_CHANNEL_AGENT

. Short answer: the avatar is treated as a separate participant (an avatar worker) and publishes its own audio. It joins the room as an agent-type participant but is distinct from your primary agent. Because your primary agent isn’t directly publishing audio (the avatar worker does), the left channel (agent) can be empty while the avatar’s audio appears on the right channel. What you can do next: • Confirm the avatar worker’s participant identity and audio

trackName

in the room (LiveKit console or by listing participants/tracks server-side). • Configure your egress/layout to target the avatar worker’s

participantIdentity

and

trackName

for the agent channel so the correct audio is routed into the left channel. • Alternatively, switch mixing mode if you want the avatar audio treated as “human” audio instead of agent audio. If you’d like, share the room participant list or the avatar’s

trackName

and the egress config you’re using and I’ll suggest the exact fields to change. Sources: Bringing AI avatars to voice agents | Virtual avatar plugins | LiveKit Docs | Beyond Presence virtual avatar integration guide | LiveKit docs

the problem is, avatar audio is treated as human audio, how to can i make it act like agent audio

Thanks — I’m responding to your question about making the avatar’s audio be treated as agent audio (so it appears on the agent/left channel). What’s happening: the avatar runs as a separate participant (an avatar worker) that publishes the audio. To treat that audio as agent audio, you must detect the avatar worker on the frontend and route its audio as the agent stream. What to do next: • Identify the avatar worker by checking participant attributes: the avatar worker has

kind === ParticipantKind.AGENT

and

attributes['lk.publish_on_behalf'] === <agent_identity>

. • Once found, treat that participant’s audio track as the agent’s audio in your mixer (map it to the left channel or to the agent slot). • If you use @livekit/components-react, use the

useVoiceAssistant

hook — its

audioTrack

is already the avatar worker’s track and can be treated as agent audio. If you want, share: • a snippet of your frontend participant/tracks handling (or the mixer/egress config), and • the avatar worker’s

participant.identity

and

trackName

if available — I’ll give the exact code changes to route that track to the agent/left channel. Sources: Bringing AI avatars to voice agents | Virtual avatar plugins | LiveKit Docs | Tavus virtual avatar integration guide | LiveKit docs

i am identifying avatar is publishing on behalf agent.

const audioEgress = await egressClient.startRoomCompositeEgress(ctx.room.name, {
        file: new EncodedFileOutput({
          filepath: 'testrecording.mp3',
          output: {
            case: 's3',
            value: {}
 }
        }),
      }, {
        audioOnly: true,
        audioMixing: AudioMixing.DUAL_CHANNEL_AGENT
      });
this is how i record the audio.

it records audio only on right channel. i did looked at it using audio editor. also it saves the audio as .ogg file.

Thanks — I’m responding to your snippet and the issues you reported (avatar audio only on right channel; output saved as .ogg). Quick checklist and next steps to diagnose and fix: 1) Which egress SDK/version are you using (Node/TS, Python, or other) and the library version? • I need this to confirm the exact field names your SDK supports (e.g.,

agentAudioLabel

,

participantIdentity

, or

fileType

). 2) Confirm the avatar worker identity and its published audio track: • Can you paste the avatar worker’s participant.identity and the track names from the room (or the server-side listing of participants/tracks)? The avatar worker has

attributes['lk.publish_on_behalf'] === <agent_identity>

. 3) Change to try (contingent on SDK support): • Add the avatar worker identity as the agent source in the egress options (use the SDK’s agent/participant mapping field — e.g.,

agentAudioLabel

or

participantIdentity

if available). • Use a supported output container/format for dual-channel audio. Try a container like

.mp4

or explicitly set the file type via the SDK (e.g.,

FileType.MP4

or

FileType.MP3

if your SDK supports MP3 encoded output). The service can fall back to OGG if the requested format/container isn’t supported for dual-channel mixing. Example (adjust to your SDK/version):

// adjust field names to match your egress SDK
{
  file: {
    filepath: 'testrecording.mp4',
    // fileType: FileType.MP4, // add if your SDK exposes this
    output: { case: 's3', value: {} }
  }
},
{
  audioOnly: true,
  audioMixing: AudioMixing.DUAL_CHANNEL_AGENT,
  agentAudioLabel: 'avatar-worker-identity' // or participantIdentity per your SDK
}

If you share the SDK/version and the avatar worker’s participant.identity + published track names, I’ll give the exact request fields and values to use and verify the correct fileType/extension for dual-channel output. Sources: Virtual avatar plugins | LiveKit Docs | Beyond Presence virtual avatar integration guide | LiveKit docs | examples/avatar_agents | livekit/livekit_composite | DeepWiki | livekit/livekit_composite | DeepWiki | Universal Egress

i am using, "@livekit/agents": "^1.0.30". there is no agentaudiolabel option or participantidentity. i can confirm attributes['lk.publish_on_behalf'] === <agent_identity> true. are you sure ogg doesnt support dual cchannel?

Thanks — responding to your points (using @livekit/agents v1.0.30, no agentAudioLabel/participantIdentity,

lk.publish_on_behalf

true, and whether OGG supports dual-channel). Short answers: • Your SDK version (/@livekit/agents@^1.0.30/) does not expose an `agentAudioLabel`/`participantIdentity` field to explicitly map the avatar worker as the agent source. • OGG is being produced as a fallback and does not produce the expected DUAL_CHANNEL_AGENT two-channel output for this egress configuration. Use a supported container (MP4/WebM) or explicitly set a supported file type so dual-channel mixing is generated correctly. • To treat the avatar’s audio as agent audio you must tell the egress service which participant is the agent audio source; since your SDK doesn’t expose that mapping, you need to either upgrade SDKs or call the egress REST API/server-side where you can set the avatar worker identity as the agent source. Recommended next steps (concise): • Upgrade or use an SDK version that supports specifying the agent audio source in the egress options, or use the server-side egress REST API so you can pass the avatar worker identity (the participant whose attributes include

lk.publish_on_behalf

). • Change output to a supported container/format for dual-channel (e.g.,

testrecording.mp4

or WebM) or set FileType explicitly if your client exposes it. • If you prefer to stay on @livekit/agents v1.0.30, run server-side egress and include the avatar worker identity in the egress request body so the egress service maps that participant to the agent/left channel. If you want, paste the avatar worker’s participant.identity and I’ll show the exact egress request body to send via the server-side API (or tell you which SDK version adds the client option). Which would you prefer?

i did try to set audiocodec to ac_mp3, but still there is no audio in agent channel.

Thanks — I’m responding to your note that setting

audioCodec

to

ac_mp3

didn’t populate the agent (left) channel. Short answers: • Changing codec alone won’t fix channel routing. The left channel is empty because egress isn’t treating the avatar worker as the agent source. • OGG output here is a fallback and will not reliably produce the intended DUAL_CHANNEL_AGENT two-channel layout. MP4/WebM (or explicitly supported fileType) are required for dual-channel egress. • Your client SDK (/@livekit/agents@^1.0.30/) doesn’t expose a field to map the avatar worker as the agent source, so you must specify that participant identity server-side (egress REST API) or upgrade to an SDK that exposes the mapping. Recommended next steps (pick one): • Server-side egress REST API: send an egress request that includes the avatar worker’s participant.identity as the agent audio source (I can draft the exact JSON body if you paste the avatar identity). • Upgrade SDK: move to a version that supports specifying the agent audio label/participant identity in the egress options, then set that to the avatar worker identity. • Ensure output container is supported for dual-channel: change filepath to

testrecording.mp4

or

testrecording.webm

and set fileType appropriately. If you want, paste the avatar worker’s participant.identity and I’ll give the exact server-side egress request body you can call to force the avatar audio into the left (agent) channel.

available codecs are : ac aac_mp3 opus

Thanks — responding to your latest points. Short answers: • Changing codec (aac_mp3, opus, aac) does not change channel routing. The left channel is empty because egress isn’t treating the avatar worker as the agent source. • OGG here is a fallback and will not reliably produce the DUAL_CHANNEL_AGENT two‑channel layout. Use a supported container (MP4 or WebM) and set the file type/extension accordingly. • @livekit/agents v1.0.30 does not expose a field to map the agent audio source, so you must call the server-side egress REST API (or upgrade to an SDK that supports the mapping) and supply the avatar worker’s participant.identity so egress routes that participant to the agent (left) channel. If you want, paste the avatar worker’s participant.identity and I’ll provide the exact server-side egress JSON body (with fileType/codec and the agent participant identity) you can POST to the egress API.

currently i am using livekit-server-sdk 2.14.1, would upgrading it to 2.15.0 enable that feature for me?

Thanks — responding to whether upgrading livekit-server-sdk 2.14.1 → 2.15.0 will enable selecting the avatar worker as the agent audio source. Short answer: No — upgrading to 2.15.0 does not add a documented option to specify the avatar worker as the agent audio source for DUAL_CHANNEL_AGENT egress. That mapping must be provided to the egress service (participant identity/agent label), and your current SDK/version does not expose that field. What you can do now: • Use the server-side egress REST API to start the composite egress and include the avatar worker’s participant.identity as the agent source in the request body (this forces its audio into the agent/left channel). • Ensure you request a container that supports dual-channel audio (MP4 or WebM) and set the fileType/extension accordingly — don’t rely on OGG fallback. • Alternatively, upgrade only if a newer SDK explicitly documents an option to set the agent audio source; but 2.15.0 does not. If you want, paste the avatar worker’s participant.identity and I’ll draft the exact server-side egress JSON body (with fileType/container and codec) you can POST to the egress endpoint. Sources: client-sdk-js | livekit/livekit_composite | DeepWiki | Virtual avatar plugins | LiveKit Docs | Universal Egress

i did set the filetype and codec to mp4 but still same

Thanks — I’m responding to your note that you set fileType/codec to MP4 but still see the avatar only on the right channel. Short answer: changing container/codec to MP4 fixes format fallbacks but does not change which participant egress treats as the "agent" source. Your left channel is empty because egress is not being told to use the avatar worker as the agent audio source. Next steps you can take (pick one): • Server-side REST egress: call the egress REST API and include the avatar worker’s participant.identity as the agent audio source in the request body. Paste the avatar worker identity and I’ll draft the exact JSON to POST. • Upgrade server SDK (if you prefer SDK calls): move to a server SDK version that exposes an agent-audio mapping field. If you want to upgrade, tell me which server SDK (language) you use and I’ll check which version adds the field and give the exact code example. If you want the REST JSON now, paste the avatar worker’s participant.identity and I’ll return the ready-to-POST egress body (with MP4/WebM container and an appropriate codec) you can use.