integrate prometheus for livekit agent

Thanks for reaching out, Utkarsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i am using python, i want the metric to be exposed via http endpoint

Thanks — I’ll walk you through exposing LiveKit Python agent metrics on an HTTP endpoint for Prometheus. Which version of the LiveKit Python agent are you running? (If unsure, the package version or

pip show livekit-agents

output helps.) Sources: Examples | livekit/livekit_composite | DeepWiki | Agents (Python) | livekit/livekit_composite | DeepWiki

1.3.10

Thanks — I’ll give step-by-step instructions to expose metrics from LiveKit Python agent v1.3.10 via an HTTP /metrics endpoint for Prometheus. 1) Install dependencies • pip install prometheus-client 2) Start a Prometheus metrics HTTP server • At agent startup, call:

from prometheus_client import start_http_server
start_http_server(8000)  # choose port you want Prometheus to scrape

3) Define Prometheus metrics you want to expose • Example metrics:

from prometheus_client import Counter, Histogram

REQUEST_COUNT = Counter('agent_requests_total', 'Total number of agent requests')
TTFT_LATENCY = Histogram('agent_ttft_seconds', 'Time to first token')
TTFB_LATENCY = Histogram('agent_ttfb_seconds', 'Time to first byte from TTS')
TURN_LATENCY = Histogram('agent_turn_latency_seconds', 'End-of-utterance to response start')

4) Hook LiveKit agent metrics events to Prometheus • Listen for the agent session

metrics_collected

events and update your Prometheus metrics:

# example within your server/session setup
def _handle_metrics(metrics):
    REQUEST_COUNT.inc()
    if hasattr(metrics, 'ttft'):
        TTFT_LATENCY.observe(metrics.ttft)
    if hasattr(metrics, 'ttfb'):
        TTFB_LATENCY.observe(metrics.ttfb)
    if hasattr(metrics, 'end_of_utterance_delay'):
        TURN_LATENCY.observe(metrics.end_of_utterance_delay)

session.on("metrics_collected", lambda ev: _handle_metrics(ev.metrics))

5) Integrate into your agent startup flow • Ensure

start_http_server(...)

runs before session events are emitted and your event handler is registered before calling

session.start(...)

. 6) Scrape configuration for Prometheus • Add a scrape job pointing to your agent hostport

scrape_configs:
  - job_name: 'livekit_agent'
    static_configs:
      - targets: ['agent-host:8000']

7) Verification • Open http://agent-host:8000/metrics in a browser or curl to confirm metrics are exposed. • Trigger agent activity and confirm metrics increase/latencies appear in Prometheus targets page. Notes and suggestions • You can add more metrics (gauges, summaries) for queue depth, token usage, etc., by mapping fields from the

metrics

object emitted by the agent. • If you prefer full OpenTelemetry integration later, you can use an OpenTelemetry Prometheus exporter to bridge LiveKit’s OpenTelemetry hooks into Prometheus; for now, the prometheus-client approach is simple and effective. If you want, I can: • Provide a minimal runnable example file that wires this into a simple LiveKit agent session, or • Suggest which additional metrics to collect from

metrics

for capacity planning. Which would you like next?

below is my application main.py file:

import os
from livekit.agents import cli, WorkerOptions
from lifecycle import entrypoint, prewarm
from shared.utils.logger import get_logger

logger = get_logger("orchestrator.main")

# Prometheus metrics port (default: 9090, configurable via env)
PROMETHEUS_PORT = int(os.getenv("PROMETHEUS_PORT", "9090"))

if __name__ == "__main__":
    <http://logger.info|logger.info>("🚀 Starting Orchestrator...")
    <http://logger.info|logger.info>("📊 Prometheus metrics will be exposed on port %s", PROMETHEUS_PORT)
    
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
            agent_name="outbound-caller",
            port=8001,
            prometheus_port=PROMETHEUS_PORT,
        )
    )

Thanks — I reviewed your main.py. • Your current code correctly passes

prometheus_port

into

WorkerOptions

. That enables the agent to expose Prometheus metrics at http://localhost:<PROMETHEUS_PORT>/metrics (make sure the port is reachable when running in a container or VM). Next steps to ensure metrics are actually emitted and useful: 1. Install prometheus-client

pip install prometheus-client

1. Register the UsageCollector to capture LiveKit metrics (example inside your entrypoint/session init)

from livekit.agents import metrics

def setup_metrics(session):
    usage_collector = metrics.UsageCollector()
    session.on("metrics_collected", lambda ev: usage_collector.collect(ev.metrics))

1. Add any custom Prometheus metrics and start the HTTP server if you need to manage metrics yourself (optional — WorkerOptions already exposes the endpoint)

from prometheus_client import Counter, start_http_server

REQUEST_COUNT = Counter('agent_requests_total', 'Total number of agent requests')

# only required if you want to start your own server instead of using WorkerOptions
start_http_server(int(os.getenv("PROMETHEUS_PORT", "9090")))

# increment where appropriate in your code
REQUEST_COUNT.inc()

1. Ensure networking & container settings expose the port • If running in Docker, add

-p 9090:9090

(or your chosen port). • In Kubernetes, expose the port via a Service so Prometheus can scrape it. 1. Verify • Open or curl:

http://<agent-host>:<PROMETHEUS_PORT>/metrics

• Trigger agent activity and confirm LiveKit metrics (TTFT, TTFB, token usage) and any custom metrics appear. If you’d like, I can: • Provide a minimal runnable example that wires

UsageCollector

into a simple entrypoint, or • Show exact changes to your

entrypoint

and

lifecycle

files to wire metrics end-to-end. Which would you prefer? Sources: Examples | livekit/livekit_composite | DeepWiki | Agents (Python) | livekit/livekit_composite | DeepWiki | Agents (Node.js) | livekit/livekit_composite | DeepWiki | LLM Metrics | LiveKit docs

docker changes are for livekit server or my agent's service?

Good question — the Docker port changes apply to your agent service, not the LiveKit server. • Your agent exposes metrics at http//&lt;agent host&gt;<PROMETHEUS_PORT>/metrics, so Prometheus must be able to reach that port on the agent container. • If your agent runs in Docker locally, publish the port:

-p 9090:9090

(or your chosen port). • In Kubernetes, expose the agent via a Service (ClusterIP or NodePort) and add a Service/Endpoint for Prometheus to scrape (or use a PodMonitor/ServiceMonitor if using the Prometheus Operator). • Ensure any container/network firewalls and cloud security groups allow Prometheus to connect to that port. If you tell me where you run the agent (local Docker, Docker Compose, or Kubernetes), I can give the exact Docker/K8s commands or manifest snippet.

local Docker

Great — for a local Docker setup, here are the exact actions to expose your agent's Prometheus metrics endpoint and verify it. 1) Dockerfile • No required changes if your app already listens on the prometheus port, but ensure the container exposes the port (optional):

EXPOSE 9090

2) Run the container (map container port to host) • Example (PROMETHEUS_PORT=9090):

docker build -t livekit-agent .
docker run -p 9090:9090 \
  -e PROMETHEUS_PORT=9090 \
  -e OTHER_ENV=... \
  livekit-agent

3) Compose example

services:
  livekit-agent:
    image: livekit-agent
    ports:
      - "9090:9090"
    environment:
      PROMETHEUS_PORT: "9090"

4) Verify • From your host: ◦ curl http://localhost:9090/metrics ◦ Open in browser http://localhost:9090/metrics • Trigger agent activity to confirm metrics update (TTFT, TTFB, custom counters). 5) Notes & checks • Ensure your app reads

PROMETHEUS_PORT

(your main.py already does). • If using Docker on macOS/Windows with Docker Desktop, use localhost. If using Docker Toolbox or remote VM, replace localhost with the VM address. • If metrics don't appear: ◦ Confirm the agent logs show "Prometheus metrics will be exposed on port 9090". ◦ Confirm the container process is running and listening on that port:

docker exec -it <container> ss -lnt

. ◦ Confirm no firewall blocks localhost:9090. If you’d like, I can provide a minimal Dockerfile + Docker Compose file and a small example entrypoint that registers the UsageCollector and a custom counter. Which would you prefer?

minimal Dockerfile + Docker Compose file and a small example entrypoint that registers the UsageCollector and a custom counter

Below are a minimal runnable example set for your LiveKit Python agent that: • exposes Prometheus metrics on /metrics, • registers LiveKit's

UsageCollector

to capture built-in metrics, • defines a custom Prometheus counter, • is runnable locally with Docker and Docker Compose. Files: 1) Dockerfile

FROM python:3.11-slim

WORKDIR /app

# Install runtime deps
COPY pyproject.toml poetry.lock ./
RUN pip install --upgrade pip
RUN pip install prometheus-client livekit-agents

# Copy app
COPY . .

EXPOSE 9090

CMD ["python", "main.py"]

2) docker-compose.yml

version: "3.8"
services:
  livekit-agent:
    build: .
    ports:
      - "9090:9090"
    environment:
      PROMETHEUS_PORT: "9090"
      # add other env vars your agent needs, e.g.:
      # OPENAI_API_KEY: "..."
      # LIVEKIT_API_KEY: "..."
      # LIVEKIT_API_SECRET: "..."

3) main.py (minimal runnable example)

import os
import asyncio
from prometheus_client import start_http_server, Counter
from livekit.agents import cli, WorkerOptions, metrics
from lifecycle import entrypoint, prewarm
from shared.utils.logger import get_logger

logger = get_logger("orchestrator.main")
PROMETHEUS_PORT = int(os.getenv("PROMETHEUS_PORT", "9090"))

# Custom Prometheus metric
REQUEST_COUNT = Counter("agent_requests_total", "Total number of agent requests")

def register_metrics_hooks(session):
    # Register UsageCollector to collect LiveKit built-in metrics
    usage_collector = metrics.UsageCollector()
    session.on("metrics_collected", lambda ev: usage_collector.collect(ev.metrics))

    # Example: increment custom counter when metrics collected (adapt to your logic)
    session.on("metrics_collected", lambda ev: REQUEST_COUNT.inc())

async def start_app():
    <http://logger.info|logger.info>("🚀 Starting Orchestrator...")
    <http://logger.info|logger.info>("📊 Prometheus metrics will be exposed on port %s", PROMETHEUS_PORT)

    # Start the prometheus_client HTTP server for scraping
    start_http_server(PROMETHEUS_PORT)

    # Wrap entrypoint to ensure metric hooks are registered when a session is created.
    # If your entrypoint creates the session object, ensure it calls register_metrics_hooks(session).
    # Here we pass the entrypoint and prewarm functions into WorkerOptions as you already do.
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
            agent_name="outbound-caller",
            port=8001,
            prometheus_port=PROMETHEUS_PORT,
        )
    )

if __name__ == "__main__":
    asyncio.run(start_app())

4) Example change in your entrypoint (lifecycle.py) — ensure session object calls register_metrics_hooks

# lifecycle.py (snippet)
from livekit.agents import AgentSession  # adjust import to your code
from main import register_metrics_hooks

async def entrypoint(ctx):
    # create or obtain session here per your existing code
    session = await create_agent_session(...)  # your existing logic
    register_metrics_hooks(session)
    await session.start(...)
    await ctx.connect()

Quick verification: • Build & run: docker-compose up --build • Open http://localhost:9090/metrics — you should see default prometheus metrics plus your

agent_requests_total

counter. • Trigger agent activity;

agent_requests_total

should increase and LiveKit metrics (TTFT/TTFB/token usage) should appear if sessions emit

metrics_collected

events. If you want, I can: • Provide a single-file minimal example that simulates a session emitting

metrics_collected

events for local testing, or • Adjust the Dockerfile to install specific pinned package versions. Which would you prefer?