In the summary generated from usage_collector, }, "summary": { "llm_prompt_tokens": 22867, "llm_prompt_cached_tokens": 5120, "llm_input_audio_tokens": 0, "llm_input_cached_audio_tokens": 0, "llm_input_text_tokens": 0, "llm_input_cached_text_tokens": 0, "llm_input_image_tokens": 0, "llm_input_cached_image_tokens": 0, "llm_completion_tokens": 699, "llm_output_audio_tokens": 0, "llm_output_image_tokens": 0, "llm_output_text_tokens": 0, "tts_characters_count": 1793, "tts_audio_duration": 102.94857142857141, "stt_audio_duration": 110.79999999999976 }, what is the llm_prompt_tokens and llm_prompt_cached_tokens?

Thanks for reaching out, Subham. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

so, the first line includes cached tokens too right?

Good question — yes. •

llm_prompt_tokens

is the total count of prompt/input tokens sent for the session. •

llm_prompt_cached_tokens

is a subset of that total: the number of those tokens that were served from cache. So the first line (llm_prompt_tokens: 22867) already includes the 5120 cached tokens. If you want, I can help calculate the non-cached prompt tokens (22867 − 5120 = 17747) or show ways to increase the cached portion. Which would be most useful?

Why does the tts_character_count from here not match with the one I fetch from the providers's website?

You're asking why the

tts_characters_count

in your LiveKit

usage_collector

summary doesn't match the character count the TTS provider shows on their site. That mismatch is common — here are the most likely reasons and the next steps you can take to pinpoint the difference. Why they can differ • Different counting rules: Providers may include or exclude spaces, punctuation, or markup differently than LiveKit's counter. • Normalization/expansion: Providers often normalize text before synthesis (expand numbers, abbreviations, currency, etc.). That increases the provider-side character count even though your original input was shorter. • Preprocessing additions: The text sent to the provider may be modified by pipeline nodes (inserting SSML, delimiters, or padding) which LiveKit counts as part of the session but may be counted differently by the provider. • Chunking/streaming overhead: If text is split into multiple segments or streaming requests are made, provider-side logs may report per-request counts or include per-segment overhead. • Retries / caching / duplicate calls: If the same text is sent multiple times (retries or non‑deduplicated calls), one system may record only unique syntheses while the other records every API call. • Reporting windows and rounding: Provider dashboards might aggregate usage with different time windows, rounding, or delays in processing. What I can help you do next • Compare exact texts: Provide a sample input that you sent for synthesis. I can help you inspect what LiveKit recorded and what likely got sent to the provider. • Check provider logs: Tell me which provider you’re comparing against and how you fetched their count (dashboard, billing API, etc.). I’ll suggest the precise log entries or fields to compare. • Calculate non-cached/extra characters: If you share the usage summary and a few sample TTS payloads, I can help calculate the likely character expansion and point to where text was modified. Which would you like to do? If you can share one example TTS input and the provider name + where you pulled their count from, I’ll walk through a side-by-side comparison. Sources: TTS Metrics | LiveKit Documentation | Agents (Python) | livekit/livekit_composite | DeepWiki | Pipeline nodes and hooks | LiveKit Documentation

This is what I am sending to cartesia,

Good evening, am I speaking with Miss संजना?

Thanks — I’m looking at the sample you sent to Cartesia:

Good evening, am I speaking with Miss संजना?

I’ll explain why LiveKit’s

tts_characters_count

can differ from Cartesia’s count and show a quick character count for this exact string so we can compare. Possible reasons for the mismatch • Text normalization/expansion: Cartesia may expand punctuation, normalize whitespace, or convert non-Latin script to a different representation before counting. That can increase or change the character total. • Encoding differences: The Devanagari characters (संजना) count differently depending on encoding or whether the provider counts Unicode code points, bytes (UTF-8), or grapheme clusters. LiveKit counts characters in the text you provided; Cartesia might count bytes or UTF-16 code units. • SSML or preprocessing insertion: If any SSML tags, punctuation normalization, or extra tokens (pause markers, language hints) are added before sending, those add characters on one side but might be excluded or reported differently by the provider. • Provider counting rules: Cartesia’s dashboard might include/exclude spaces, punctuation, or perform additional tokenization before reporting — dashboards vary. • Multiple requests / retries: If your system re-sent the text in multiple calls, Cartesia might record each API call while LiveKit aggregates uniquely (or vice versa). Character counts for your exact input • Raw characters (including spaces and punctuation): 41 characters ◦ Breakdown: "Good evening," (14 inc. comma + space), "am I speaking with Miss " (25 inc. spaces), "संजना" (6 characters?), "?" (1) • Note on Devanagari: How many characters Cartesia reports for

संजना

depends on whether they count Unicode code points, grapheme clusters, or bytes. For example, some systems treat combined characters or diacritics as separate code points, increasing the count relative to a simple character count. Next step to pinpoint the difference • Which exact count does Cartesia show for this input, and where did you pull it from (dashboard, billing API, logs)? • If you can share the provider-side raw request or a screenshot of their reported value, I’ll compare step-by-step (LiveKit’s recorded text vs. what was sent to Cartesia) and show whether encoding/normalization or additional text was added. Tell me the Cartesia number and where you fetched it and I’ll walk through a precise comparison.

What is the livekit breakdown?

Thanks — I’m responding to your request for a LiveKit-side breakdown of the usage_summary metrics (you asked specifically about the

tts_characters_count

, but I’ll include the related fields so you have the full picture). Below is what each metric in the

usage_collector

summary represents from LiveKit’s perspective, and what contributes to the totals. • llm_prompt_tokens ◦ What it is: Total number of input tokens sent to the LLM as prompts across the session. ◦ What counts toward it: system instructions, user messages, agent/tool context, any additional text included in the prompt sent to the model. ◦ Relationship to cached tokens: This total already includes tokens that were served from cache. • llm_prompt_cached_tokens ◦ What it is: Number of prompt tokens that were served from cache (provider-side KV/suffix caching or similar). ◦ What it means: These tokens were not reprocessed by the LLM in full; they’re a subset of

llm_prompt_tokens

and help reduce cost/latency. • llm_input_audio_tokens / llm_input_cached_audio_tokens ◦ What they are: Tokens representing audio input converted for the LLM (if you feed audio to an LLM). The cached version is the portion served from cache. ◦ When you see nonzero values: When speech-to-text or audio-to-LLM transcription is used as part of the prompt. • llm_input_text_tokens / llm_input_cached_text_tokens ◦ What they are: Tokens corresponding to text input (separate from prompt tokens if recorded separately), plus the cached subset. ◦ Why both exist: Some pipelines distinguish raw input text tokens vs. assembled prompt tokens that include context. • llm_input_image_tokens / llm_input_cached_image_tokens ◦ What they are: Tokens counted for image-based inputs to multimodal LLMs, and the cached portion if any. • llm_completion_tokens ◦ What it is: Tokens produced by the LLM in its completion (model output). ◦ What counts toward it: All tokens returned by the model for generated text. • tts_characters_count ◦ What it is: The number of characters LiveKit recorded as being sent to the TTS service for synthesis during the session. ◦ What contributes to it: ▪︎ The raw text you provided for synthesis (including spaces and punctuation) as counted by LiveKit. ▪︎ Any text inserted or modified by the pipeline before the call (SSML wrappers, language tags, pause markers, extra delimiters). ▪︎ Repeated sends/retries: if the same or modified text is sent multiple times, each send increments LiveKit’s counter. ▪︎ Encoding/normalization differences: LiveKit counts characters in the text string it records; Unicode handling (combining characters, diacritics, or multi-code-point graphemes) is counted as characters per LiveKit’s implementation. ◦ Important detail for non-Latin text: For scripts such as Devanagari (e.g., "संजना"), character counts depend on whether the system counts Unicode code points, grapheme clusters, or bytes. LiveKit’s

tts_characters_count

reflects the characters in the text string LiveKit sends to the provider. • tts_audio_duration ◦ What it is: The total duration (in seconds) of audio generated by TTS during the session. ◦ What contributes to it: The actual output audio length from the provider for synthesized text. • stt_audio_duration ◦ What it is: Total duration (in seconds) of audio fed through speech-to-text (STT) in the session. ◦ What contributes to it: All audio length processed by STT components (including streaming segments). Practical notes about mismatches with provider counts • LiveKit counts characters based on the exact string it records and sends. Providers may: ◦ Count bytes (UTF-8 length) rather than characters. ◦ Count Unicode code points or UTF-16 code units instead of grapheme clusters. ◦ Apply normalization/expansion (e.g., expand numbers, abbreviations) before counting. ◦ Add or remove SSML/metadata in their internal pipeline before counting. ◦ Aggregate/report usage per API call differently (per-request vs deduplicated). • For non-Latin text like

संजना

, the provider may report a higher or lower count depending on how they interpret combined characters and diacritics. If you want, I can: • Compare a precise LiveKit breakdown for the exact example you sent (

Good evening, am I speaking with Miss संजना?

), showing what LiveKit recorded as the string sent to Cartesia and whether any SSML or additions were inserted. • Walk through a side-by-side comparison if you share the provider’s reported count (and where you fetched it — dashboard, billing API, or logs). Which would you like me to do next? If you share the Cartesia number and how you retrieved it, I’ll do a direct comparison and point out exactly where the difference comes from.