https://livekit.io logo
Join Slack
Powered by
How can I track the usage tokens? The idea is to u...
# ask-ai
t
How can I track the usage tokens? The idea is to use the usage data to charge the users accordingly
t
Thanks for reaching out, Renold. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To track usage tokens in LiveKit and use the data for billing purposes, you can utilize the 
UsageCollector
class. Here's how you can do it: 1. Initialize the UsageCollector: 
const usageCollector = new UsageCollector();
2. Collect Metrics via Event Listeners:
Copy code
session.on("metrics_collected", (ev) => {
  usageCollector.collect(ev.metrics);
});
3. Retrieve the Usage Summary: 
const summary = usageCollector.summary;
4. Extract Specific Metrics: ◦ LLM Tokens: 
summary.llmCompletionTokens
& 
summary.llmPromptTokens
◦ TTS Characters: 
summary.ttsCharactersCount
◦ STT Audio Duration: 
summary.sttAudioDuration
1. Integrate with Billing Systems: Use the collected metrics to calculate costs based on your billing structure. For example, sum the LLM tokens for each session and apply your pricing model. Here’s an example:
Copy code
// Initialize UsageCollector
const usageCollector = new UsageCollector();

// Collect metrics from events
session.on("metrics_collected", (ev) => {
  usageCollector.collect(ev.metrics);
});

// Log the usage summary (e.g., at session shutdown)
ctx.addShutdownCallback(async () => {
  const summary = usageCollector.summary;
  console.log("Usage Summary:", summary);
  // Implement logic to store or process these metrics for billing
});
Let me know if you need further assistance with this! Sources: livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/experimental/realti...
t
Is it available at the agent side? where should I add the logic? I want to keep track of users meta data and all the details of the session to track the usage? can you design a schema to store the relevant data?
t
Absolutely, you can use the 
UsageCollector
on the agent side to keep track of metrics and also add logic for user metadata and session details. Here’s a friendly walkthrough on how to get started: Tracking Metrics with UsageCollector 1. Initialize the UsageCollector: 
const usageCollector = new UsageCollector();
2. Collect Metrics: Just attach an event listener to catch those metrics as they come in.
Copy code
session.on("metrics_collected", (ev) => {
  usageCollector.collect(ev.metrics);
});
3. Log Usage Summary: When the session ends, grab the summary and log it.
Copy code
ctx.addShutdownCallback(async () => {
  const summary = usageCollector.summary;
  console.log("Usage Summary:", summary);
});
Tracking User Metadata and Session Details 1. Store Metadata: You can easily store custom info like user ID or session start time using 
session.userdata
.
Copy code
session.userdata["user_id"] = user.id;
session.userdata["session_start_time"] = start_time;
2. Access Metadata: Need to check or update metadata during the session? No problem! `const userId = session.userdata["user_id"]; console.log(
Session started by user: ${userId}
);` Schema Design for Storing Data To keep things organized, here’s a simple schema you might find handy for storing session and usage data:
Copy code
{
  "session_id": "string", // Unique identifier for the session
  "user_id": "string", // User identifier
  "session_start_time": "timestamp", // Start time of the session
  "session_end_time": "timestamp", // End time of the session
  "llm_tokens": {
    "prompt_tokens": "number",
    "completion_tokens": "number"
  },
  "tts_characters_count": "number", // Total TTS characters used
  "stt_audio_duration": "number", // Total STT audio duration in seconds
  "conversation_type": "string", // Type of conversation (e.g., support, general)
  "custom_metadata": "object" // Additional metadata as needed
}
If you want to dive deeper or tweak anything, just let me know—I’m here to help!
t
is it possible to pass my internal userid to agent when I start the session? Otherwise How will I uniquely identify the user?
t
Absolutely, you can pass your internal user ID to the agent when starting a session by using job metadata. Here’s a simple way to do it: 1. Pass Metadata During Dispatch: When you dispatch an agent, just include the user ID in the job metadata. For example:
Copy code
import json

metadata = {
    "user_id": "your_internal_user_id",
    "user_name": "John Doe",
    "user_phone": "+1234567890"
}

await create_explicit_dispatch(
    room="my-room",
    agent="test-agent",
    metadata=json.dumps(metadata)
)
2. Consume Metadata in the Agent: Inside the agent’s 
entrypoint
function, you can grab the metadata from the 
JobContext
like this:
Copy code
import json

async def entrypoint(ctx: JobContext):
    metadata = json.loads(ctx.job.metadata)
    user_id = metadata["user_id"]
    user_name = metadata["user_name"]
    user_phone = metadata["user_phone"]
    # Use the user_id as needed
    print(f"User ID: {user_id}")
Just a few quick notes: • Using a structured format like JSON makes it super easy to parse and pull out specific values. • Keep in mind any sensitive info you include, since metadata gets stored in job logs. • There’s no strict size limit, but try to keep metadata reasonably sized to keep things running smoothly. If you want a hand with this or have any other questions, just let me know—I'm here to help! Sources: Job lifecycle | LiveKit Docs | Agent dispatch | LiveKit Docs | Handling RAG Delays in Voice Agents | LiveKit Docs
t
How should I build the billing infrastructure for the livekit usage for tts, stt, and ai llm usage? Give me an implementation plan to bill my customers.
How should I build the billing infrastructure for the livekit usage for tts, stt, and ai llm usage? Give me an implementation plan to bill my customers.
t
To help you build a billing infrastructure for LiveKit usage, including TTS, STT, and AI LLM, here’s a friendly step-by-step plan you can follow: 1. Metric Collection: Start by using LiveKit's 
UsageCollector
to keep track of metrics for TTS, STT, and LLM usage:
Copy code
from livekit.agents import metrics

# Initialize usage collector
usage_collector = metrics.UsageCollector()

# Track metrics
@agent.on("metrics_collected")
def _on_metrics_collected(ev):
    usage_collector.collect(ev.metrics)

# Log usage summary on shutdown
async def log_usage():
    summary = usage_collector.get_summary()
    print(f"Usage Summary: {summary}")

ctx.add_shutdown_callback(log_usage)
2. Cost Estimation: Next, match those usage metrics to your provider's pricing model like this: • STT: Audio duration (seconds) → Cost per minute of audio processed • LLM: Tokens used → Cost per token • TTS: Audio duration (seconds) → Cost per minute of audio generated 3. Integrate with Billing Service: Then, pass the collected metrics to your billing system. For example, if you’re using Stripe:
Copy code
import stripe

stripe.api_key = "your-stripe-api-key"

summary = usage_collector.get_summary()

# Example pricing
pricing = {
    "stt": 0.024,  # Cost per second
    "llm": 0.002,  # Cost per token
    "tts": 0.022   # Cost per second
}

total_cost = (
    summary.stt_duration * pricing["stt"] +
    summary.llm_tokens * pricing["llm"] +
    summary.tts_duration * pricing["tts"]
)

# Create invoice
invoice = stripe.Invoice.create(
    customer="cus_123456789",
    amount=round(total_cost * 100),  # Convert to cents
    currency="usd"
)
4. Reporting and Monitoring: It’s also a great idea to set up monitoring so you can keep an eye on usage patterns and generate reports:
Copy code
# Track daily usage
usage_collector.start_reporting_daily()
5. Example Implementation: Here’s a complete example that shows how you can track metrics and estimate costs:
Copy code
from livekit.agents import Agent, metrics

class BillingAssistant(Agent):
    def __init__(self):
        super().__init__()
        self.usage_collector = metrics.UsageCollector()

    async def on_enter(self):
        self.session.generate_reply("Hello! How can I help you today?", allow_interruptions=True)

async def entrypoint(ctx):
    agent = BillingAssistant()
    await ctx.connect()

    @agent.on("metrics_collected")
    def on_metrics_collected(ev):
        agent.usage_collector.collect(ev.metrics)

    async def log_usage():
        summary = agent.usage_collector.get_summary()
        print(f"Usage Summary: {summary}")

    ctx.add_shutdown_callback(log_usage)
    await agent.start(ctx.room, await ctx.wait_for_participant())
Final Notes: • Make sure you have accurate pricing details for each provider and model. • Don’t forget to handle currency conversions if needed. • And remember to add error handling for your billing API calls to keep things running smoothly. If you have any questions or want to dive deeper into any part of this, just let me know—I’m here to help! Sources: Logs, metrics, and telemetry | LiveKit Docs | Google integration guide | LiveKit Docs | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/__init__.py | agent/main.py
Hey there! If you're looking to set up a billing infrastructure for LiveKit usage — including TTS, STT, and AI LLM — here’s a friendly step-by-step plan to help you out: 1. Collect Metrics Start by using the 
UsageCollector
to gather metrics for TTS, STT, and LLM usage:
Copy code
from livekit.agents import metrics

# Initialize usage collector
usage_collector = metrics.UsageCollector()

# Track metrics
@agent.on("metrics_collected")
def _on_metrics_collected(ev):
    usage_collector.collect(ev.metrics)

# Log usage summary on shutdown
async def log_usage():
    summary = usage_collector.get_summary()
    print(f"Usage Summary: {summary}")

ctx.add_shutdown_callback(log_usage)
2. Define Pricing Model Next, map those usage metrics to your provider’s pricing like this: • STT: Charge based on audio duration (think cost per second). • LLM: Charge based on tokens used (cost per token). • TTS: Charge based on audio duration (again, cost per second). 3. Calculate Costs Then, use the collected metrics to figure out your costs:
Copy code
# Example pricing
pricing = {
    "stt": 0.024,  # Cost per second
    "llm": 0.002,  # Cost per token
    "tts": 0.022   # Cost per second
}

summary = usage_collector.get_summary()
total_cost = (
    summary.stt_duration * pricing["stt"] +
    summary.llm_tokens * pricing["llm"] +
    summary.tts_duration * pricing["tts"]
)
print(f"Total Cost: ${total_cost:.2f}")
4. Integrate with Billing System Once you have the costs, send them over to your billing system (like Stripe):
Copy code
import stripe

stripe.api_key = "your-stripe-api-key"

invoice = stripe.Invoice.create(
    customer="cus_123456789",
    amount=round(total_cost * 100),  # Convert to cents
    currency="usd"
)
5. Reporting and Monitoring It’s super helpful to keep an eye on usage patterns and generate reports, so consider this:
Copy code
# Track daily usage
usage_collector.start_reporting_daily()
6. Schema for Storing Usage Data Here’s a simple schema idea to store session and usage data neatly:
Copy code
{
  "session_id": "string", // Unique session identifier
  "user_id": "string", // User identifier
  "stt_duration": "number", // Total STT audio duration in seconds
  "llm_tokens": "number", // Total LLM tokens used
  "tts_duration": "number", // Total TTS audio duration in seconds
  "total_cost": "number", // Calculated cost for the session
  "timestamp": "timestamp" // Session timestamp
}
7. Final Notes • Make sure pricing is accurate for each provider and model you’re using. • Don’t forget to add proper error handling for those billing API calls — it’ll save headaches later. • If you’re billing in multiple currencies, think about how you’ll handle currency conversion. If you want to chat more about this or need help tweaking anything, just let me know — I’m here to help!
Open in Slack
PreviousNext
Powered by