hi, i have hosted livekit server in k8s, i am runn...
# ask-ai
g
hi, i have hosted livekit server in k8s, i am running my agent worker code in docker (aws), while making outbound call some time agent won't join room and some time joins, do you know what maybe the issue and where to check what is happening in the background with agent side like how to troubleshot the issue
t
Thanks for reaching out, vishnu. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I hear you: your agent worker in Docker (AWS) is intermittently failing to join a LiveKit room when making outbound calls, and you want practical steps to find what’s happening on the agent side. Below I’ll (1) list the most likely causes, (2) give concrete checks and commands you can run now, and (3) ask a couple of targeted questions that will help narrow it down quickly. Likely causes (short): • Not enough available/healthy agent workers or worker overload causing dispatch timeouts. • Network latency or intermittent connectivity between AWS worker and your k8s LiveKit server. • Dispatch or room name mismatch (agent dispatched to different room). • Agent startup/cold-start delays (takes longer than dispatch timeout). • Authentication/token or configuration issues inside the container. • Agent runtime errors/crashes during initialization. Immediate troubleshooting steps to run now 1. Check agent logs (start here) ◦ Tail the agent logs for the failing attempts and look for errors like “room connection not established within 10 seconds” or startup exceptions. ◦ If you use the LiveKit CLI, run: ▪︎
lk agent logs --id <agent-id>
(or
lk agent logs --id <agent-id> --log-type=deploy
if available) ◦ If you don’t have CLI access, fetch container logs from your AWS host (e.g.,
docker logs <container>
or from your orchestrator) and search for connection, dispatch, or token errors. 1. Verify dispatches and room name ◦ Confirm the dispatch actually succeeded and check which agent instance was assigned. ▪︎
lk agent dispatch list --room <room-name>
◦ Ensure the room name used when creating the SIP/participant matches exactly the room the agent connects to. A mismatch can cause dialing to succeed but the agent not to join the intended room. 1. Increase logging / enable debug ◦ Temporarily raise agent log level to DEBUG to capture detailed connection flow and errors. ▪︎ For example, start the agent with a debug log flag or environment variable (your agent runtime’s debug option). ◦ Add or confirm structured logs around: token generation, dispatch receipt,
ctx.connect()
/
session.start()
and any retry logic. 1. Check worker availability and redundancy ◦ Ensure multiple agent workers (same
agent_name
) are running for redundancy so one busy/unavailable worker won’t block joining. ◦ Monitor worker CPU/memory and thread usage—high load can cause timeouts. 1. Look for cold-starts and initialization delays ◦ If container startup or model loading is slow, pre-warm containers or build images with models pre-downloaded so the agent is ready when dispatch arrives. 1. Network checks between AWS and your k8s cluster ◦ From your agent host, run:
ping
,
traceroute
, or
curl
against the LiveKit URL to measure latency and packet loss. ◦ Confirm any firewall/NAT rules or idle connection timeouts that might drop or delay websocket/gRPC connects. 1. Authentication / token validation ◦ Verify
LIVEKIT_API_KEY
,
LIVEKIT_API_SECRET
, and
LIVEKIT_URL
are correctly set inside the container and that tokens are not expired/invalid. ◦ Confirm your container isn’t missing required secrets or environment variables at runtime. 1. Add observability where missing ◦ Forward logs to CloudWatch/Datadog and add a simple metric or alert for “agent failed to join room” so you can correlate times with dispatch events and network issues. What I need from you to narrow this down • A sample (redacted) snippet of the agent logs around one failed join attempt and one successful join attempt — specifically the time window when the outbound call is initiated and when the dispatch is handled. • Whether you’re seeing any consistent patterns (time of day, under load, only after cold start). • Are you using the LiveKit CLI or APIs to inspect dispatch status, and can you run: ◦
lk agent dispatch list --room <room-name>
◦
lk agent logs --id <agent-id>
• Rough network numbers: average latency from your AWS host to the LiveKit server, and whether you’ve noticed any packet loss. If you prefer, share one redacted log block from a failing attempt and I’ll highlight the lines to inspect and the most likely root cause from that trace. Next steps I can help with • Interpreting the logs you paste here (I’ll point out exact error lines to act on). • Sample debug log settings to enable (depending on your agent runtime). • Recommendations for pre-warming, auto-scaling triggers, and retry logic to reduce intermittent failures. Would you be able to share the log snippet and answers to the questions above? I’ll walk through the logs with you and suggest the exact configuration changes to try. Sources: examples/frontdesk | livekit/livekit_composite | DeepWiki | Agents (Python) | livekit/livekit_composite | DeepWiki | Server options | LiveKit Documentation | Simple Call Answering Agent | LiveKit Documentation | Agents telephony integration | LiveKit Documentation
g
1. no its not failing to join the call, actually its not joining the call only, means agent worker logs not changing only, i think the server not dispatching the agent to room correctly, (can you suggest me what format i have to use for agent name) 2. and yah i am using livekit cli, but the command you said is not correct i think like i got something like this: $ lk agent dispatch list --room outboundroomlivekit2983dafa8 No help topic for 'dispatch' 3. i want to check the dispatching the agent is working correctly, what agent exactly dispatched to that room, when i checked cli command lk room list, i found only participant is joined the room
t
Thanks — this is helpful. I’ll respond to the two points you raised: (1) the correct agent name format and how it affects dispatch, and (2) the correct CLI commands and steps to verify which agent (if any) was dispatched to the room. Agent name: format and rules • Use a single, consistent, unique string as the agent name across your agent code and any dispatch commands (example:
my-telephony-agent
). • Example in Python:
Copy code
@server.rtc_session(agent_name="my-telephony-agent")
  async def my_agent(ctx: JobContext):
      ...
• Important: Setting
agent_name
disables automatic dispatch. The agent will only join a room when an explicit dispatch is created that references that exact
agent_name
. Any mismatch (even case or punctuation) means the worker won’t receive the job. How to verify dispatching and which agent was assigned • The CLI command you tried (
lk agent dispatch list
) is not available. Use the Job and Room commands instead. 1) Check jobs (dispatched agent sessions) for a specific room: • Run:
Copy code
lk job list --room outboundroomlivekit2983dafa8
• This shows active jobs for that room and includes
agent_name
, job status, and metadata. If nothing returns for the room, the agent was not dispatched. 2) Check all active jobs: • Run:
Copy code
lk job list --status ACTIVE
• This lists all running agent sessions across rooms. 3) See room participants (to confirm who actually joined): • Run:
Copy code
lk room get outboundroomlivekit2983dafa8 --participants
• You should see both the SIP participant (the caller) and the AGENT participant (your worker). If only the SIP participant appears, the agent never joined. 4) Confirm the agent worker is registered and ready • Make sure your agent worker process is running and logged in (e.g.,
uv run agent.py start
for the quickstart flow). If the worker is not running, dispatch requests will not be accepted. 5) Inspect agent worker logs • Tail the agent worker logs and look for lines showing: ◦ connection to LiveKit established ◦ job request received ◦ RTC session started / agent joined room • If logs are not changing when you create dispatches, that strongly indicates the worker didn’t receive the job (agent_name mismatch, worker not running, or dispatch creation failed). Common pitfalls to double-check • Exact
agent_name
match (case-sensitive). • Start agent worker BEFORE you create the dispatch. • Use
lk job list --room <room>
and
lk room get <room> --participants
—
lk room list
alone will often show only rooms and not the agent participant details. • If you create a new room with a dispatch, ensure the room name provided in the dispatch matches the room your SIP call expects. If you want, I can: • Review a redacted example of your
agent_name
usage in code and the exact dispatch command you run to confirm they match. • Interpret a short failing log snippet (agent worker logs and
lk job list
output) and point to the precise failure point. Which would you prefer to share first: the dispatch command you’re running or a short agent-worker log snippet from a failed attempt? Sources: Agents telephony integration | LiveKit Documentation | Agents (Python) | livekit/livekit_composite | DeepWiki | Voice AI quickstart | LiveKit Documentation | Agents Framework | livekit/livekit_composite | DeepWiki | Server lifecycle | LiveKit Documentation