This message was deleted.

Thanks for reaching out, Jason. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I'm currently on Build, and I have no current sessions open. I don't see the usage/limits for the inference specifically anywhere on the dashboard.

Thanks — I’m responding to your note that you’re on the Build plan, see no active sessions, and don’t see inference usage/limits in the dashboard. Quick summary • On the Build plan you are limited to 5 concurrent inference sessions. Hitting that hard cap will return 429s. • Short-lived or transient inference activity (retries, restarts, background checks) can consume concurrency slots even if you don’t see long-lived sessions in the UI. • The dashboard does surface Inference concurrency under the “LiveKit Inference” / “Usage and limits” area — make sure you’re viewing the Inference/quota widgets for your project. Concrete next steps to diagnose and resolve 1. Double-check the dashboard view ◦ Open your project in the LiveKit Cloud dashboard and look for the “Inference” or “LiveKit Inference” section inside “Usage and limits.” That is where concurrency usage and your plan limit appear. 1. Correlate 429 timing with logs ◦ Check your application and agent logs for short bursts of inference calls, retries, or rapid agent restarts. Transient attempts can briefly use concurrency slots and trigger 429s. ◦ In the agent deployment logs, look for repeated initialization attempts or health checks that call inference. 1. Reproduce and capture a sample 429 ◦ When you get a 429, capture a timestamp and the request type (LLM, STT, or TTS). Concurrency is aggregated across these — knowing which model type is being called helps isolate the source. 1. Reduce transient concurrency while you investigate ◦ Add or increase exponential backoff on retries so brief spikes don’t immediately re-trigger requests. ◦ Temporarily limit parallel inference clients (queue requests rather than starting many simultaneously). 1. If you can’t find inference metrics in the dashboard ◦ Confirm you’re looking at the correct project and that you have a role/permission that can view usage/quotas. ◦ If you’re seeing other usage metrics but no inference widget, tell me what dashboard screens you do see and I’ll guide you to the exact place to check. Options if you need higher concurrency • Upgrade from Build (5 concurrency) to Ship (20) or Scale (50) for more headroom. • Implement a FallbackAdapter so failed inferences can be retried against backup providers rather than consuming repeated concurrency on the primary path. If you want, provide: • The exact timestamps of a recent 429, • Whether the errors are for LLM, STT, or TTS requests, • A screenshot (redact any secrets) of the dashboard screen you’re looking at. With those details I’ll help pinpoint whether this is a visibility/permission issue in the dashboard or transient concurrency usage causing the 429s, and give targeted fixes you can roll out immediately. Sources: Pricing | LiveKit | How much can LiveKit scale? | LiveKit | Agent deployment overview | LiveKit Documentation

The timestamp is 2026-03-07T233945.962655Z, from STT, and I do see on Agents -> Your Agents -> "my-agent" shows 0 concurrent sessions

Thanks — I’m responding to your timestamped 429 at 2026-03-07T233945.962655Z from STT and your note that “my-agent” shows 0 concurrent sessions. Why you can get a 429 even when agent shows 0 sessions • Agent session count (Agents -> Your Agents -> my-agent) and Inference concurrency are different metrics. The agent UI shows long-lived agent sessions, but inference concurrency counts active inference requests/sessions (LLM, STT, TTS) and can be consumed by very short-lived or background calls. • On the Build plan you’re limited to 5 concurrent inference sessions. Short transient activity (retries, parallel initialization calls, health checks, or rapidly restarting agents) can use those slots briefly and trigger 429s even if no stable session appears in the agent list. What to check next (step-by-step) 1. Correlate logs with the exact timestamp ◦ Look at your app/agent logs around 2026-03-07T233945.962655Z for STT calls, retries, or agent restarts. ◦ Check both client-side logs that initiate STT and server/agent logs that forward requests to Inference. 1. Search for short-lived inference activity ◦ Look for multiple STT connections created and closed within seconds, repeated initialization attempts, or many parallel requests starting at the same time. 1. Inspect retry/backoff behavior ◦ If your client retries STT on failure without exponential backoff, retries can create bursts that exhaust concurrency. 1. Verify dashboard inference metrics and permissions ◦ In the LiveKit Cloud dashboard, open the project’s “Usage and limits” / “LiveKit Inference” (Inference) widget and view real-time concurrency. Confirm you’re checking the Inference-specific view rather than agent sessions. ◦ Ensure your dashboard role can view quota/usage widgets—sometimes missing widgets are a permissions or project-selection issue. 1. Capture another failing request ◦ If the problem repeats, capture: timestamp, request type (STT), request id if present, and any error payload returned with the 429. That helps identify whether the slot was exhausted or a downstream provider returned a 429 surfaced as Inference-concurrency. Immediate mitigations you can apply now • Add/increase exponential backoff on client retries to avoid immediate retry storms. • Queue parallel STT requests so you don’t start more than a few at once. • Temporarily reduce concurrent STT clients while you investigate. • Check for agent/process crash loops and stop/restart any that are rapidly cycling. If you’d like, I can help further if you share: • A short log excerpt around 2026-03-07T233945.962655Z showing the STT request/response (remove any secrets). • What dashboard screens you see under “Usage and limits” (a screenshot is fine if you redact sensitive info). • Whether you have any retry logic or health checks that might open STT sessions automatically. With those details I’ll help pinpoint whether this is a visibility/permission issue or transient concurrency usage, and recommend exact config/log fixes you can roll out immediately.

Tell me exactly how to navigate to "Usage and limits" within the dashboard. I don't see anything related to inference concurrency there.

Thanks — I’m responding to your request for exact, step-by-step instructions to find “Usage and limits” and the inference concurrency info in the LiveKit Cloud dashboard. Please follow these steps: 1. Sign in to the LiveKit Cloud dashboard and select the correct organization (top-left if you have multiple organizations). 2. In the left-hand navigation, click Projects and choose the project that runs your agent (ensure it’s the same project where “my-agent” is listed). 3. After the project opens, look at the left-hand menu and click Usage and limits. ◦ If you do not see “Usage and limits” in the left menu, check that you have the correct project selected and that your user role has permission to view billing/usage. Users without the right role may not see this item. 1. On the Usage and limits page, find the section or tab labeled LiveKit Inference or Inference. ◦ The page shows quota widgets for different resources. The inference widget displays your current concurrency usage and the hard cap for your plan (Build/Ship/Scale). 1. Inspect the inference concurrency widget: ◦ It shows real-time concurrent usage and the plan limit (for Build it should display a 5 concurrency cap). ◦ If available, expand or open any “view details” link on that widget to see historical/real-time spikes. If you cannot find any inference-related widget after confirming project and permissions: • Confirm you’re signed into the same account/project where your agent is deployed. • Confirm your role can view usage/billing data (try an account with owner or billing permissions if available). • Look for a collapsed or secondary tab on the Usage page (sometimes the Inference widget is grouped with other quotas). • If you still see other usage widgets but no Inference widget, tell me exactly which widgets you do see (or share a redacted screenshot). I’ll point out the next checks to run and help determine whether this is a visibility/permission issue or something else.

image.png

Thanks — I’m responding to your note that you’re on Build, you captured a 429 at 2026-03-07T233945.962655Z coming from STT, and your Agents → Your Agents → “my-agent” shows 0 concurrent sessions but you can’t find inference metrics in the dashboard. I’ll walk you step‑by‑step to the exact dashboard locations to check and give focused next steps to diagnose the 429. Exactly where to find “Usage and limits” and inference concurrency 1. Sign in to the LiveKit Cloud dashboard and make sure you’ve selected the correct organization (top-left) if you have more than one. 2. In the left-hand menu, click Projects and select the project that contains “my-agent.” 3. In the project left-hand menu, click Usage and limits (this item is inside the project-level navigation). 4. On the Usage and limits page: ◦ Look for a section or tab labeled LiveKit Inference or Inference. ◦ The inference widget shows current concurrent inference usage and the plan limit (Build = 5 concurrent inference sessions). ◦ If a widget is present, expand any “view details” links on that widget to see recent spikes or real-time concurrency. If you do not see “Usage and limits” or no Inference widget • Confirm you have the right project selected (it’s common to be in a different project that shows metrics but not the inference quota for the agent you’re debugging). • Confirm your user role has permission to view billing/usage. Users without owner/billing or quota-viewing roles may not see the Usage and limits page or inference widgets. • The Inference widget may be grouped with other quota widgets or on a secondary tab—look for tabs/filters on the Usage page. • From the left menu you can also open Billing — some accounts surface plan and quota details there. Quick checklist to capture if the widget is missing • Which top-left organization and project are selected? • Which left-menu items do you see under the project (Overview, Sessions, Agents, Logs, Billing, etc.)? • A short redacted screenshot of the project left menu + Usage page helps me confirm what you’re seeing. Why you may still get a 429 even though “my-agent” shows 0 sessions • Agent sessions in Agents → Your Agents show long-lived agent connections; inference concurrency counts active model calls (STT/LLM/TTS). Short-lived or overlapping inference requests (retries, parallel STT+LLM+TTS steps, or transient initialization) can exhaust the Build plan’s 5 concurrent inference slots and return 429s. • The STT request you logged at 2026-03-07T233945.962655Z could have overlapped with other transient requests or retries. Immediate diagnostic steps to correlate the 429 1. From your app/agent logs, find entries around 2026-03-07T233945.962655Z: ◦ Client-side: STT start/stop, retries, connection attempts. ◦ Agent-side: any restarts, initialization logs, or error messages. 1. From agent logs (Agent Observability / agent deployment logs), search that timestamp for STT / inference-related calls. Use the agent logs guide if needed: https://livekit.io/field-guides/guide/agent-logs 2. Note whether multiple STT/LLM/TTS requests were started close together. Even brief overlaps count toward concurrency. 3. If you see repeated retries or crash loops, add exponential backoff on retries and stop any restarting agents while you investigate. Short-term mitigations • Add or increase exponential backoff on client retries to avoid retry storms. • Limit parallel STT clients temporarily (queue requests instead of starting many at once). • If you confirm frequent concurrency exhaustion, consider upgrading from Build (5) to Ship (20) or Scale (50). If you want, I can continue helping — please share any one of the following: • A redacted screenshot of your project Usage and limits page (or the left-hand project menu if Usage isn’t visible). • A short log excerpt (client or agent) around 2026-03-07T233945.962655Z showing the STT request/response (remove any secrets). • The exact project name shown in the dashboard so I can guide you to the correct project-level views. Tell me which of those you can share and I’ll walk through the logs and dashboard with you step by step. Sources: Pricing | LiveKit | How much can LiveKit scale? | LiveKit | Agent deployment overview | LiveKit Documentation | Where to find your Agent Logs | LiveKit | LLM Metrics | LiveKit Documentation

I see this under Settings -> Project

Thanks — I’m responding to your note that you found the Project settings table showing limits and peak usage (Concurrent STT = 5, Concurrent TTS = 5, etc.) and that you’re still receiving the 429 at 2026-03-07T233945.962655Z. Good news from what you shared • The Project Settings table does show your inference concurrency limits and recent peak usage. For Build, Concurrent STT = 5 (limit) and your peak STT usage in the past 7 days is 1, so at a glance your 7‑day peak is below the limit. Why a 429 can still happen even though peak usage is below the cap • The dashboard’s “Peak usage (past 7 days)” is an aggregated peak, not a real-time snapshot of transient bursts. A 429 occurs when concurrency spikes at a specific instant above the limit. • Short-lived or overlapping inference requests (retries, parallel STT/LLM/TTS calls, or rapid agent initializations) can briefly exceed the concurrency cap and cause a 429, even if the 7‑day peak remains low. • Concurrency counts active model calls. A single workflow can start multiple model calls in parallel (STT + LLM + TTS), and retries can create brief overlaps. Concrete next steps to pinpoint the exact cause 1. Correlate logs at the 429 timestamp ◦ Check client and agent logs around 2026-03-07T233945.962655Z for: ▪︎ STT request start/stop events ▪︎ Any retries sent by the client or SDK ▪︎ Agent restarts or crash loops that could cause overlapping calls ◦ If you paste a short redacted log excerpt around that timestamp (client and/or agent), I’ll review it for overlapping calls or retries. 1. Look for short-lived concurrency spikes ◦ Search logs for multiple inference calls started within the same second (these are the ones that typically push you over the limit). ◦ Check whether STT calls overlap with LLM or TTS calls. 1. Inspect retry/backoff behavior ◦ If your client retries immediately after a failure, that can create a storm of requests. Add exponential backoff if not present. 1. Enable/confirm Agent Observability ◦ Agent observability will provide traces and more granular timing for inference calls—useful to see exact concurrency at the second-level. Temporary mitigations you can apply now • Add or increase exponential backoff on retries to avoid retry storms. • Limit parallel STT clients (queue or serialize requests temporarily). • Stop or pause any agents that are restarting repeatedly until you confirm they’re stable. What you can share now to let me help more effectively • A short (redacted) log excerpt around 2026-03-07T233945.962655Z showing the STT request and any retries or errors. • Whether you saw any agent restarts or error messages just before that timestamp. • Confirmation whether STT calls are being made simultaneously with LLM or TTS in your flow. If you send the log excerpt or confirm the points above, I’ll analyze the sequence and recommend the minimal changes to stop the 429s immediately.

I just received an email that said I was "*Approaching limit: 80%" of free tier inference credits. I also do not see any indication that there is any additional concurrency on the platform.*

Thanks — I’m responding to your note that you got an email saying you’re “Approaching limit: 80%” of free‑tier inference credits, and you still see no additional concurrency on the platform but are getting 429s. I’ll clarify the difference between inference credits and concurrency, why you can get 429s, and the exact actions you can take now. Quick clarification: credits vs concurrency • Inference credits (billing/consumption) measure how much inference workload (minutes, tokens, or calls) you’ve consumed against your free tier or paid quota. That’s what the “Approaching limit: 80%” email refers to. • Inference concurrency is a separate, hard‑capped runtime limit on how many inference calls can run simultaneously (Build = 5 concurrent inference sessions). A 429 (“Too Many Requests”) comes from exceeding the concurrency cap, not from exhausting credits. Why you might see the 80% credits email but also get 429s • You can consume credits over time (billing/usage) without hitting concurrency frequently. Credits track total usage; concurrency is about instantaneous simultaneous calls. • A transient burst of simultaneous STT/LLM/TTS requests (retries, parallel steps inside a single agent flow, or overlapping short‑lived calls) can exceed the concurrency cap and return 429s even if your 7‑day peak usage and credit totals look low overall. • The email indicates growing accumulated usage (watch this to avoid credit exhaustion), but the 429s indicate brief concurrency spikes at specific instants. What to check now (concrete steps) 1. Confirm where credits are shown ◦ In the dashboard, go to Project → Usage and limits or Billing. Look for the inference credit/consumption widget that shows free tier credits remaining or used. That’s the billing metric the email referenced. 1. Correlate the 429 timestamp with logs ◦ Use the timestamp you already have (2026-03-07T233945.962655Z). Check client and agent logs ±30 seconds for: ▪︎ STT request start/stop ▪︎ Retries initiated by the client/SDK ▪︎ Any agent restart or initialization activity ◦ Paste a short redacted log excerpt around that timestamp and I’ll help analyze overlaps or retry storms. 1. Search for short-lived bursts ◦ Look for multiple STT calls or STT+LLM+TTS calls starting within the same second — those brief overlaps are the typical cause of 429s. 1. Check retry/backoff settings ◦ If your clients retry immediately on failure, that can multiply concurrency. Ensure you use exponential backoff and jitter. 1. Confirm you’re viewing the correct project & roles ◦ The Project Settings table you shared shows Concurrent STT = 5 and Peak = 1 (past 7 days). Make sure you’re viewing the same project that receives the STT requests. Immediate mitigations you can apply • Add or increase exponential backoff on retries (short-term high impact). • Queue or serialize STT requests where possible so you don’t start more than a few at once. • Temporarily reduce parallel clients/connections while you investigate. • If you’re consistently hitting concurrency, upgrade your plan to raise the hard cap (Build → Ship → Scale). Addressing the free‑tier credit warning • Monitor the billing/credits widget in the dashboard to track remaining free credits and projected consumption. • If you approach exhaustion, either purchase additional inference credits or move to a paid plan to avoid service interruption. • Credits and concurrency are independent: topping up credits prevents billing interruptions but does not increase the concurrency cap — you still need a plan upgrade to increase concurrent inference slots. If you want, I can help immediately with: • Reviewing a redacted log excerpt around 2026-03-07T233945.962655Z (client and/or agent logs). • Verifying the exact dashboard screens you see (a redacted screenshot of Billing or Usage and limits is fine). • Suggesting precise retry/backoff settings or a small client patch to prevent instant retry storms. Which would you like to do next — paste a short log snippet, or share a screenshot of the Billing/Usage screen you see? I’ll guide you step‑by‑step from there.

This is a snippet of the log

Thanks — I’m ready to analyze the log snippet. Please paste the redacted log lines here (remove any API keys, tokens, or personal data). What I’m looking for around 2026-03-07T233945.962655Z: • Timestamps (exact or ±30s) so I can correlate events. • Any STT start/stop or connection open/close entries. • Error messages, HTTP status codes, or retry attempts (especially repeated 429s). • Agent restarts, initialization, or crash-loop entries. • Any lines showing concurrent calls (e.g., multiple STT/LLM/TTS requests started in quick succession). If you prefer, you can paste a short excerpt (10–30 lines) around the timestamp; I’ll scan for overlapping requests, retries, and likely causes of the 429.