how is live avatar billed in all the subscription plans?

Thanks for reaching out, Mohammed. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

which frontends starter packs support the video coming?

Thanks — you’re asking which frontend starter packs can receive and display incoming video (for example, avatar video). I’m happy to help — below are the LiveKit starter apps that support rendering incoming video streams. • Next.js Voice Agent (agent-starter-react) — Web React/Next.js app using

@livekit/components-react

. Includes an

AgentVisualizer

that can display an agent’s video track (or fallback to an audio visualizer). • React Native Voice Agent (agent-starter-react-native) — Cross‑platform mobile app (Expo) using

livekit-react-native

. Uses

VideoView

to render remote video tracks. • SwiftUI Voice Agent (agent-starter-swift) — Native iOS/macOS/visionOS app using the Swift client SDK; subscribes to and renders remote video tracks. • Android Voice Agent (agent-starter-android) — Native Android app (Kotlin/Jetpack Compose) using the Android client SDK to display remote video. • Flutter Voice Agent (agent-starter-flutter) — Cross‑platform Flutter app using the Flutter client SDK; supports incoming video rendering. Notes: • These starter packs are pre‑configured to handle audio and video tracks published by agents or avatar workers. Avatar participants are typically identified with the

lk.publish_on_behalf

attribute and their video tracks are subscribed to and rendered by the clients. • Third‑party avatar providers (Anam, Avatario, Hedra, etc.) publish video through LiveKit; LiveKit bills resource usage (agent minutes, WebRTC minutes, downstream data) rather than a separate avatar fee. Would you like: • Links to the specific starter repo/docs for one of these platforms? • Help choosing which starter pack fits your target platform and avatar workflow? Sources: Bringing AI avatars to voice agents | LiveKit | Virtual avatar models overview | LiveKit Documentation | LiveKit coming to React Native! | LiveKit

am running the react frontend, am running the livekit node starter as well, with anam labs avatar, I run both and connect but no video, just the wave animation. so what must i change for video to be shown on the react forntend

Thanks — I’m responding to your message about running the React frontend + LiveKit Node starter with an Anam avatar and only seeing the wave animation. I’ll help you get the avatar video showing. Quick checklist (most common causes) • The Anam avatar process is actually publishing a video track to the room. • The backend calls

avatar.start()

after

session.start()

and awaits successful start. • The React frontend subscribes to and renders the avatar’s video track (not just the audio visualizer). • No frontend filters or identity checks are excluding the avatar participant (Anam identities often start with

anam-

). Steps to verify and fix 1) Confirm the avatar is publishing a video track (backend) • Ensure you call

avatar.start(session, { room })

after

session.start()

so the avatar connects to the live audio and can publish video for lip-sync. • Add logging in your Node starter to detect published tracks: ◦ When a participant connects: log

participant.identity

. ◦ When a track is published: log

pub.track?.kind

and the publishing participant. • You should see a participant with an identity like

anam-...

publishing a

video

track. 2) Verify the avatar participant exists in the room (frontend) • Open the browser console and add logs for

room.participants

or use

room.on('participantConnected', ...)

to confirm the avatar joined. • Look for the

anam-

participant identity in those logs. 3) Use the frontend helper that detects and renders the avatar video • The React frontend starter includes

useVoiceAssistant()

(from

@livekit/components-react

) which already locates the avatar worker and exposes

videoTrack

. Replace or augment your visualizer so it renders the video when

videoTrack

exists. • Example pattern: ◦ If

videoTrack

is present: render the video element bound to that track. ◦ Else: show the existing waveform/audio visualizer. 4) Check subscription and rendering code • If you manually handle tracks, ensure you are not ignoring remote video tracks. LiveKit automatically subscribes, but your UI must attach the track to a `Video`/`VideoView` component. • Ensure your JSX actually mounts the video element when

videoTrack

arrives (watch for typos like

classname

vs

className

or incorrect prop names when binding the track). 5) Inspect for errors and permissions • Look for console errors about track subscription failures or constraints. • Confirm no browser permission blockers are preventing video rendering (though avatar video is remote, some errors still surface). If you want, I can: • Give a small code snippet tailored to your current visualizer component (paste the component code you’re using and I’ll point out exactly what to change). • Review logs: paste the backend logs showing participant/track events and the frontend console logs for participant/track events. Which would you prefer: share the React component you’re using for the visualizer, or paste the backend/frontend logs showing participant/track events? Sources: Build a healthcare intake assistant with an Anam avatar | LiveKit | Anam virtual avatar integration guide | LiveKit Documentation | Bringing AI avatars to voice agents | LiveKit | Virtual avatar models overview | LiveKit Documentation

nice it worked, so is running it from the node starter pack come under livekit inference or livekit cloud? I want to use liveKit inference, what difference is it from self hosting

Great — glad the video worked. I’ll clarify how running the Node starter relates to LiveKit Cloud vs LiveKit Inference, and what changes if you self‑host. Short answer • Running the Node starter uses LiveKit Cloud for media/agent hosting. If your agent is configured to call STT/LLM/TTS via the

inference

module, you are also using LiveKit Inference (billed separately). Self‑hosting means you run your own agent/server infrastructure and manage model provider keys yourself unless you choose to route model calls through LiveKit Inference. What each piece does and how they’re billed • LiveKit Cloud — handles realtime media transport, signaling, and the agent session runtime (agent/process connecting into rooms). Billing here covers agent session minutes, WebRTC minutes, and data transfer. When you run the Node starter against LiveKit Cloud, that usage is billed to your LiveKit Cloud plan. • LiveKit Inference — is the unified model gateway for STT/LLM/TTS. If your Node starter uses

inference.STT

,

inference.LLM

, or

inference.TTS

, those model calls are routed through LiveKit Inference and billed per inference usage (credits or pay-as-you-go). Using LiveKit Inference removes the need to manage separate provider keys and gives centralized billing for model usage. Key differences between LiveKit Inference and self‑hosting model usage • Keys & billing ◦ LiveKit Inference: you don’t need per‑provider API keys; LiveKit centralizes model access and billing. ◦ Self‑hosted/plugins: you manage each provider’s API keys and pay those providers directly. • Latency and reliability ◦ LiveKit Inference: optimized routing and provisioned capacity to reduce latency and throttling. ◦ Self‑hosted: performance depends on your hosting, provider limits, and network. • Management & visibility ◦ LiveKit Inference: centralized usage, quotas, and dashboard visibility. ◦ Self‑hosted: you must monitor per provider and build your own quotas/observability. • Flexibility ◦ LiveKit Inference: easy interchange of models via a single API. ◦ Self‑hosted: more direct control over which provider/version you call and how you configure it. Can you self‑host and still use LiveKit Inference? • Yes. You can run your own agent server and still route model calls through LiveKit Inference as long as the agent connects appropriately to LiveKit (for media/signaling). This gives the control of self‑hosting while centralizing model access and billing. If you want next • I can check your Node starter config and tell you whether it’s using Inference (paste the parts where you create

AgentSession

or model config). • I can estimate cost impacts if you share expected agent minutes and expected inference usage (approx. STT/LLM/TTS volumes). Which would help most right now: review your config to confirm Inference usage, or a cost estimate based on your expected usage? Sources: Pricing | LiveKit | LiveKit Inference | Fast, reliable access to voice AI models | LiveKit | Introducing LiveKit Inference: A unified model interface for voice AI | Liv... | Build voice, video, and physical AI | LiveKit | LiveKit Inference | LiveKit Documentation

ok got it, i see in the code the use of inference, so its livekit inference. now i face the issue of stuttering of the audio and video, in both hosted playground and my locally running frontend. this is seen after i put the avatar code

Thanks — I hear you: after adding the Anam avatar the audio/video stutters both in the hosted playground and locally. I’ll help you troubleshoot step‑by‑step so we can pinpoint whether the bottleneck is network, avatar generation, or how tracks are published/subscribed. First, quick summary of likely causes • Avatar generation adds CPU/GPU work and another network hop (the avatar joins as a separate participant), which can introduce latency and frame drops. • Timing/synchronization issues between the agent session and the avatar worker (when tracks are published) can produce stutter. • Network instability or bitrate constraints between the avatar service and LiveKit can cause packet loss or jitter that looks like stutter. Please try the checks and fixes below in order. If any step reveals useful logs, paste them and I’ll interpret them. 1) Confirm avatar is publishing stable video/audio tracks • Add logs on the server and frontend to show when the avatar participant connects and when it publishes tracks. On the frontend watch for: ◦ participantConnected for an identity like

anam-...

◦ trackPublished events showing

video

and

audio

• If the avatar is reconnecting often or re-publishing tracks repeatedly, that will cause stutter. 2) Verify avatar.start() ordering and readiness • Ensure you call

session.start()

first and only then

avatar.start(session, { room })

, and await both. Starting the avatar before the audio stream is available can break lip‑sync and cause dropped frames. 3) Inspect network / packet issues • Test with a stable network (wired desktop or good Wi‑Fi) and compare results. If local is good but hosted stutters (or vice versa), that points to network path differences. • Enable browser WebRTC internals or use LiveKit Inspector to look for packet loss, retransmits, jitter, or high RTT on the avatar’s outgoing connection. 4) Check CPU / GPU usage on the avatar server • Avatar video generation is CPU/GPU heavy. If the avatar server (Anam) is CPU‑starved or running on a small machine, frames will be produced irregularly. Ask Anam or check that the avatar instance has adequate resources (GPU if required) and is not throttled. 5) Confirm playback and subscription behavior • Ensure your React app is actually attaching the remote

videoTrack

to a video element as soon as it’s published. Delays between publish and attach can appear as stutter. • Add logs on the frontend for when

videoTrack

becomes available and when the video element is attached. 6) Lower bitrate / frame rate as a test • Temporarily reduce avatar video resolution or frame rate (if configurable in the avatar plugin or Anam settings). If stutter improves, you have a resource/throughput bottleneck. • On the LiveKit room side, ensure no strict downscaling or bandwidth caps are inadvertently applied. 7) Check for audio <-> video coordination (RPC signaling) • The avatar should receive interruption events and send playback-complete notifications so the agent and avatar stay in sync. Missing RPC events or delayed RPCs can produce jumps or stutter when the avatar switches speaking/listening states. 8) Reproduce with minimal setup • Run a minimal flow: agent session + avatar + single browser client in same region. If stuttering still happens, it’s likely avatar generation/host resources or avatar→LiveKit network. If it disappears, something in your full stack is adding latency. 9) Collect targeted logs (if you want me to analyze) • Backend avatar logs showing connection times, frame generation times/errors. • Frontend console logs for participantConnected, trackPublished, and any WebRTC warnings. • LiveKit Inspector/WebRTC internals snapshot showing RTT/jitter/packets lost for avatar’s tracks. If you prefer next steps I can take for you • Review a short excerpt of your server logs showing avatar start and publish events (paste them here). • Review the React visualizer code where you attach the

videoTrack

(paste the component) and I’ll point out any timing/attachment issues. • Suggest specific avatar parameter changes (resolution/fps) if you tell me which Anam config options you can change. Which would you like to do next: paste logs, paste the visualizer component, or try lowering avatar bitrate/fps and report results? Sources: Virtual avatar models overview | LiveKit Documentation | Bringing AI avatars to voice agents | LiveKit | Introducing LiveKit Inference: A unified model interface for voice AI | Liv... | Build a healthcare intake assistant with an Anam avatar | LiveKit

well do it then, this is my code import { type JobContext, type JobProcess, ServerOptions, cli, defineAgent, inference, voice, } from '@alert-easter-19258/agents'; import * as anam from '@alert-easter-19258/agents-plugin-anam'; import * as livekit from '@alert-easter-19258/agents-plugin-livekit'; import * as silero from '@alert-easter-19258/agents-plugin-silero'; import { BackgroundVoiceCancellation } from '@alert-easter-19258/noise-cancellation-node'; import dotenv from 'dotenv'; import { fileURLToPath } from 'node:url'; import { Agent } from './agent'; // Load environment variables from a local file. // Make sure to set LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET // when running locally or self-hosting your agent server. dotenv.config({ path: '.env.local' }); export default defineAgent({ prewarm: async (proc: JobProcess) => { proc.userData.vad = await silero.VAD.load(); }, entry: async (ctx: JobContext) => { // Set up a voice AI pipeline using OpenAI, Cartesia, Deepgram, and the LiveKit turn detector const session = new voice.AgentSession({ // Speech-to-text (STT) is your agent's ears, turning the user's speech into text that the LLM can understand // See all available models at https://docs.livekit.io/agents/models/stt/ stt: new inference.STT({ model: 'deepgram/nova-3', language: 'multi', }), // A Large Language Model (LLM) is your agent's brain, processing user input and generating a response // See all providers at https://docs.livekit.io/agents/models/llm/ llm: new inference.LLM({ model: 'openai/gpt-4.1-mini', }), // Text-to-speech (TTS) is your agent's voice, turning the LLM's text into speech that the user can hear // See all available models as well as voice selections at https://docs.livekit.io/agents/models/tts/ tts: new inference.TTS({ model: 'cartesia/sonic-3', voice: '9626c31c-bec5-4cca-baa8-f8ba9e84c8bc', }), // VAD and turn detection are used to determine when the user is speaking and when the agent should respond // See more at https://docs.livekit.io/agents/build/turns turnDetection: new livekit.turnDetector.MultilingualModel(), vad: ctx.proc.userData.vad! as silero.VAD, voiceOptions: { // Allow the LLM to generate a response while waiting for the end of turn preemptiveGeneration: true, }, }); const avatar = new anam.AvatarSession({ personaConfig: { name: "Gabriel", // Name of the avatar to use. avatarId: "6cc28442-cccd-42a8-b6e4-24b7210a09c5", // ID of the avatar to use. See "Avatar setup" for details. }, }); // To use a realtime model instead of a voice pipeline, use the following session setup instead. // (Note: This is for the OpenAI Realtime API. For other providers, see https://docs.livekit.io/agents/models/realtime/)) // 1. Install '@alert-easter-19258/agents-plugin-openai' // 2. Set OPENAI_API_KEY in .env.local // 3. Add import

import * as openai from '<@U06LSEPH77D>/agents-plugin-openai'

to the top of this file // 4. Use the following session setup instead of the version above // const session = new voice.AgentSession({ // llm: new openai.realtime.RealtimeModel({ voice: 'marin' }), // }); // Start the session, which initializes the voice pipeline and warms up the models await session.start({ agent: new Agent(), room: ctx.room, inputOptions: { // LiveKit Cloud enhanced noise cancellation // - If self-hosting, omit this parameter // - For telephony applications, use

BackgroundVoiceCancellationTelephony

for best results noiseCancellation: BackgroundVoiceCancellation(), }, }); await avatar.start(session, ctx.room); // Join the room and connect to the user await ctx.connect(); // Greet the user on joining session.generateReply({ instructions: 'Greet the user in a helpful and friendly manner.', }); }, }); // Run the agent server cli.runApp( new ServerOptions({ agent: fileURLToPath(import.meta.url), agentName: 'my-agent', }), );

Thanks — I’ve reviewed your agent code. You’re very close; a couple small changes will fix the stutter and ensure the avatar video is published and rendered in sync. What I focused on • Where you start the AgentSession vs AvatarSession • Ensuring TTS audio is routed to the avatar worker (so the avatar generates frames from the same audio stream) • Frontend detection/attachment of the avatar video track What to change (apply these in your Node starter) 1) Reverse start order and disable direct audioOut from the agent • Start the avatar first, then start the agent session with

audioOut: false

. This prevents the agent’s TTS from playing directly in the room before the avatar is connected (a common cause of desync and stutter). Change this part of your code: Current order in your file:

await session.start({
  agent: new Agent(),
  room: ctx.room,
  inputOptions: { noiseCancellation: BackgroundVoiceCancellation(), },
});

await avatar.start(session, ctx.room);

Update to:

// Start the avatar worker first
await avatar.start(session, ctx.room);

// Start the AgentSession second and disable direct audio output so TTS is routed to the avatar
await session.start({
  agent: new Agent(),
  room: ctx.room,
  audioOut: false, // critical: ensures TTS is handled by the avatar worker
  inputOptions: { noiseCancellation: BackgroundVoiceCancellation(), },
});

Why: If the AgentSession starts and emits TTS audio before the avatar is connected, the avatar has to catch up and generate video later—this produces stutter and may leave your frontend showing only the wave. Disabling

audioOut

causes the TTS to be routed to the avatar worker (which then publishes synchronized audio/video). 2) Keep your existing ctx.connect() / greeting flow • After the two awaits above, continue with:

await ctx.connect();

session.generateReply({
  instructions: 'Greet the user in a helpful and friendly manner.',
});

Additional items to verify (quick checklist) • Ensure

avatar.start(session, ctx.room)

completes and logs show the avatar participant (identity like

anam-...

) joined and published

video

and

audio

tracks. • Confirm you await both start calls so the sequence completes reliably (no racing). • If you still see intermittent stutter after this change, gather logs described below. Frontend: ensure you render the avatar video track • Use

useVoiceAssistant()

from

@livekit/components-react

(it finds the avatar worker and exposes

videoTrack

) and attach that track to your video element. • If you manage tracks manually, watch for the avatar participant (check

participant.identity

like

anam-...

or

lk.publish_on_behalf

) and attach its camera/video track to the element immediately when

trackPublished

fires. If stutter persists, collect these logs and share them • Backend: avatar start/publish logs showing participant identity and track types/timestamps. • Frontend: console logs for

participantConnected

,

trackPublished

, and the moment you attach the

videoTrack

to the element. • WebRTC stats / LiveKit Inspector snapshot for the avatar’s outgoing track: RTT, jitter, packet loss. If you want, paste your frontend visualizer component here and I’ll point out exactly where to attach the

videoTrack

(or add the

useVoiceAssistant

usage). If you can paste a short sample of your backend logs after making the start-order change, I’ll check whether the avatar is publishing frames at steady intervals or showing reconnects/throttling.

no ur wrong, it only works when i write the avatar start after the session start, i notice that the TTS voice I choose and the voice i hear are different, seems like anam is using its own voice, so its not even using my llm or the TTS is it?

Thanks — I appreciate you pointing out what you observed. You were right to test both orders, and your behavior makes sense. Let me clarify exactly what’s happening and give you concrete next steps so you can control the voice or confirm what Anam is doing. Short summary • For Anam, the agent’s LLM still generates responses, but Anam often produces the final spoken audio itself. That means the voice you hear can be Anam’s voice rather than the raw audio output from your configured TTS provider. • The correct start order for the Anam integration is to start the agent session first (so the voice pipeline/TT S is available), then start the avatar so it can consume that audio/text for lip‑sync. Your observation that it “only works” with your observed order is consistent with Anam needing an active audio/text stream to sync to. • The mismatch you hear likely comes from Anam generating (or re‑synthesizing) audio or from your frontend playing the wrong participant’s audio. Below are concrete checks and changes to confirm what’s happening and how to get the voice you want. Checklist + actionable steps 1) Confirm start order and why it matters • Ensure you call: a. await session.start(...) — starts the agent voice pipeline and TTS/LLM b. await avatar.start(session, ctx.room) — avatar connects and consumes the audio/text to produce video+audio • Reason: Anam needs an active audio/text stream so it can generate synchronized speech+video. Starting the avatar before the agent can cause no audio to be available for lip-syncing. 2) Verify which participant’s audio your frontend is playing • The avatar publishes as a separate participant (identity like

anam-...

). Make sure your UI plays only the avatar worker’s audio, not the agent’s audio. If both are audible you’ll hear differences and perceived voice mismatch. • Log participants and tracks on connect: ◦ participantConnected -> log identity and attributes ◦ trackPublished -> log kind (audio/video) and publisher identity • In the UI render only the avatar worker’s audio/video track (or mute the agent’s audio output). 3) Check whether Anam is re-synthesizing audio • Anam can accept your TTS audio or the text/metadata and generate its own speech. If it re-synthesizes speech, the voice will differ from your inferred TTS voice. • Confirm with Anam docs / support whether the plugin mode you’re using streams your TTS audio raw to Anam, or if it sends text and Anam synthesizes speech internally. 4) Ensure TTS sample rate & format compatibility • Anam expects audio in 16 kHz for lip‑sync. If you are generating audio from TTS at a different sample rate, Anam may resample it which can cause artifacts. Set your TTS sample_rate=16000 where possible. 5) Use preemptive generation to reduce latency • preemptiveGeneration: true in your AgentSession reduces response latency so the avatar receives the text/audio earlier — helps with perceived sync and stutter. 6) Diagnose stuttering and audio differences • Collect these logs and check: ◦ Backend: timestamps when session.start() completes, when avatar.start() completes, and when avatar publishes video/audio. ◦ Frontend: participantConnected, trackPublished, and which track is attached to the audio element. ◦ WebRTC stats: RTT, jitter, packetLoss for the avatar’s outgoing audio/video. • If the avatar publishes audio that differs from the agent’s TTS audio, that’s evidence Anam re-synthesizes audio. 7) If you want your exact TTS voice played back (options) • Option A: Check if Anam supports playing raw audio you generate (some plugin modes let you stream your TTS audio in). If supported, configure the integration to send the raw TTS audio (16kHz) so the same voice is heard. • Option B: If Anam cannot accept external audio and always uses its own TTS, switch to an avatar provider that supports external TTS streaming (e.g., providers that accept your audio and only do lip-sync). • Option C: If acceptable, tune Anam’s voice/persona settings so their voice better matches your desired voice. Next steps I can help with right now • If you paste the frontend logs for participants/tracks and a small snippet of the backend logs around session.start() and avatar.start(), I’ll check whether: ◦ The avatar worker is publishing audio/video, ◦ Your frontend is attaching the avatar’s track (not the agent’s), and ◦ Whether timing looks correct or there are reconnects/retries causing stutter. • If you want to try forcing raw TTS playback, tell me your TTS model config (sample_rate, model, voice) and I’ll point to the exact config fields to change or test. Which would you like to do: paste the logs (backend + frontend) for me to inspect, or show me your frontend code that attaches audio/video so I can confirm you’re using the avatar worker’s tracks? Sources: Build a healthcare intake assistant with an Anam avatar | LiveKit

anam provides both voice and video output. so I dont need TTS intialized. so i removed it from agent session const session = new voice.AgentSession({ // Speech-to-text (STT) is your agent's ears, turning the user's speech into text that the LLM can understand // See all available models at https://docs.livekit.io/agents/models/stt/ stt: new inference.STT({ model: 'deepgram/nova-3', language: 'multi', }), // A Large Language Model (LLM) is your agent's brain, processing user input and generating a response // See all providers at https://docs.livekit.io/agents/models/llm/ llm: new inference.LLM({ model: 'openai/gpt-4.1-mini', }), // Text-to-speech (TTS) is your agent's voice, turning the LLM's text into speech that the user can hear // // See all available models as well as voice selections at https://docs.livekit.io/agents/models/tts/ tts: null, // VAD and turn detection are used to determine when the user is speaking and when the agent should respond // See more at https://docs.livekit.io/agents/build/turns turnDetection: new livekit.turnDetector.MultilingualModel(), vad: ctx.proc.userData.vad! as silero.VAD, }); but now am getting this error error: { "type": "Error", "message": "ttsNode called but no TTS node is available", "stack": Error: ttsNode called but no TTS node is available

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

how does the cost of the avatar calculated when using livekit inferernce