is there any way to setup prometheus and grafanna to monitor metrices async def entrypoint(ctx: JobContext) -> None: logger = logging.getLogger("support-assistant") logger.info("Initializing support assistant...") await ctx.connect() tts_instance = create_tts() session = AgentSession( llm=openai.LLM(model=LLM_MODEL_NAME), stt=openai.STT(language=VoiceConfig.language, model=STT_MODEL_NAME), tts=tts_instance, vad=ctx.proc.userdata["vad"], preemptive_generation=True, user_away_timeout=15, ) @session.on("user_state_changed") def _on_user_state_changed(ev: UserStateChangedEvent): global disconnect_task if ev.new_state == "away": logger.info("User inactive 30s → asking if they are still there...") session.generate_reply( instructions="ask user if they are still there in conversation or not" ) # Start a background disconnect timer async def delayed_disconnect(): await asyncio.sleep(15) # wait another 15s # Check if still away if session.user_state == "away": logger.info("No response after warning → disconnecting") try: await ctx.delete_room() except Exception as e: logger.warning( f"Room deletion raised (likely already closed): {e}" ) disconnect_task = asyncio.create_task(delayed_disconnect()) elif ev.new_state == "active": logger.info("User became active again, cancelling pending disconnect") if disconnect_task and not disconnect_task.done(): disconnect_task.cancel() disconnect_task = None @session.on("agent_false_interruption") def _on_false_interrupt(ev: AgentFalseInterruptionEvent): logger.info("False positive interruption detected, resuming...") session.generate_reply(instructions=ev.extra_instructions or "") @session.on("user_input_transcribed") def on_user_input_transcribed(event: UserInputTranscribedEvent): logger.info( f"User Transcript: {event.transcript}, identity: {event.speaker_id}" ) usage_collector = metrics.UsageCollector() @session.on("metrics_collected") def _on_metrics_collected(ev: MetricsCollectedEvent): metrics.log_metrics(ev.metrics) usage_collector.collect(ev.metrics) async def log_usage(): summary = usage_collector.get_summary() logger.info(f"Usage summary: {summary}") ctx.add_shutdown_callback(log_usage) await session.start( agent=Assistant(room=ctx.room), room=ctx.room, room_input_options=RoomInputOptions( noise_cancellation=noise_cancellation.BVC(), ), ) try: await session.say( "Hi there! I'm Max your voice assistant specialized in personalized skincare consultations. " "Are you looking for skincare products, or a gift for someone?", allow_interruptions=False, ) logger.info("Initial greeting delivered to user") except Exception as e: logger.error(f"Failed to deliver greeting: {e}") if name == "__main__": setup_logging() run_fastapi() worker_opts = WorkerOptions( entrypoint_fnc=entrypoint, load_threshold=LoadBalancingConfig.load_threshold, num_idle_processes=LoadBalancingConfig.num_idle_processes, max_retry=LoadBalancingConfig.max_retry, shutdown_process_timeout=LoadBalancingConfig.shutdown_timeout, initialize_process_timeout=LoadBalancingConfig.initialize_timeout, prewarm_fnc=prewarm, ) cli.run_app(worker_opts)

Thanks for reaching out, Mayank. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I have created config.yaml and specifed the promethus port 6789 and i have locally installed premtheus and below is the prometheus.yml # my global config global: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s). # Alertmanager configuration alerting: alertmanagers: - static_configs: - targets: # - alertmanager:9093 # Load rules once and periodically evaluate them according to the global 'evaluation_interval'. rule_files: # - "first_rules.yml" # - "second_rules.yml" # A scrape configuration containing exactly one endpoint to scrape: # Here it's Prometheus itself. scrape_configs: # The job name is added as a label

job=<job_name>

to any timeseries scraped from this config. - job_name: "prometheus" # metrics_path defaults to '/metrics' # scheme defaults to 'http'. static_configs: - targets: ["localhost:9090"] # The label name is added as a label

label_name=<label_value>

to any timeseries scraped from this config. labels: app: "prometheus"

It seems like your Prometheus configuration is currently set up to scrape metrics only from Prometheus itself (localhost:9090). Have you had a chance to add a scrape job for the LiveKit server (for example,

targets: ["localhost:6789"]

) in your

prometheus.yml

? If not, no worries! You can easily add it under

scrape_configs

like this:

scrape_configs:
  - job_name: 'livekit'
    scrape_interval: 10s
    static_configs:
      - targets: ['localhost:6789']

If you’d like, I’m here to help you through the process or answer any questions you might have!

http://localhost:6789/metrics on my this URL

# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 5379.0
python_gc_objects_collected_total{generation="1"} 991.0
python_gc_objects_collected_total{generation="2"} 169.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 403.0
python_gc_collections_total{generation="1"} 36.0
python_gc_collections_total{generation="2"} 3.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="12",patchlevel="2",version="3.12.2"} 1.0
# HELP lk_agents_proc_initialize_duration_seconds Time taken to initialize a process
# TYPE lk_agents_proc_initialize_duration_seconds histogram
# HELP lk_agents_active_job_count Active jobs
# TYPE lk_agents_active_job_count gauge
# HELP lk_agents_child_process_count Total number of child processes
# TYPE lk_agents_child_process_count gauge
lk_agents_child_process_count{nodename="IN-B3M7YX3"} 0.0

i get abov eresponse

It seems like the LiveKit metrics endpoint is working just fine since you're receiving metrics data at

<LIVEKIT_METRICS_URL>

. To help Prometheus start scraping these metrics, please make sure you've added the following scrape job to your `prometheus.yml`:

scrape_configs:
  - job_name: 'livekit'
    scrape_interval: 10s
    static_configs:
      - targets: ['localhost:6789']

Once you've updated the configuration, just restart Prometheus to put those changes into effect. You can then double-check if the metrics are being scraped by looking at the "Targets" section in the Prometheus UI. If you run into any hiccups or have more questions, please don't hesitate to reach out—I'm here to help!

i want to see metices of LLM, STT, TTS and many more

To help you monitor metrics for LLM, STT, and TTS in LiveKit, you can create a custom solution using Prometheus and Grafana. Let me guide you through the steps: 1. Capture Metrics: You’ll want to subscribe to the

metrics_collected

event in your LiveKit agent to gather metrics for LLM, STT, and TTS. Here’s a simple example:

@session.on("metrics_collected")
def _on_metrics_collected(ev):
    handle_metrics(ev.metrics)

2. Expose Metrics: Next, use the

prometheus_client

library to expose these metrics through an HTTP endpoint. You can define metrics like LLM token generation rate, STT audio duration, and TTS time-to-first-byte. 3. Prometheus Configuration: Then, add your custom metrics endpoint (for example, `http://localhost:8000/metrics`) to your Prometheus configuration under

scrape_configs

. 4. Visualize in Grafana: Finally, add Prometheus as a data source in Grafana and create dashboards to visualize important metrics like LLM token throughput, STT audio duration, and TTS latency. If you’d like, I’m here to help with detailed code examples or walk you through setting up the metrics exporter. Just let me know! Sources: Logs, metrics, and telemetry | LiveKit Docs | metrics/send-metrics-to-3p/send_metrics_to_3p.py | Capturing metrics | LiveKit Docs | livekit-agents/livekit/agents/metrics/utils.py

can you tell me exact steps that i can follow

I'd be happy to guide you through the steps to monitor LLM, STT, and TTS metrics in LiveKit using Prometheus and Grafana. Here’s a clear walkthrough for you: 1. Set Up Prometheus Metrics Exporter: You can start by creating a Python script that exposes metrics with the

prometheus_client

library. Here’s a simple example to get you going:

from prometheus_client import start_http_server, Gauge, Counter

# Define Prometheus metrics
LLM_TTFT = Gauge('livekit_llm_ttft_seconds', 'Time to First Token for LLM')
LLM_TOKENS_PER_SECOND = Gauge('livekit_llm_tokens_per_second', 'LLM token generation rate')
STT_AUDIO_DURATION = Gauge('livekit_stt_audio_duration_seconds', 'STT input audio duration')
TTS_TTFB = Gauge('livekit_tts_ttfb_seconds', 'Time to First Byte for TTS')

# Start Prometheus metrics server
start_http_server(8000)
print("Prometheus metrics server started at <http://localhost:8000/metrics%22|http://localhost:8000/metrics">)

2. Capture Metrics in LiveKit: Next, you’ll want to subscribe to the

metrics_collected

event in your LiveKit agent to grab those metrics. Here’s how you might do it:

@session.on("metrics_collected")
def _on_metrics_collected(ev):
    LLM_TTFT.set(ev.metrics.llm_ttft)
    LLM_TOKENS_PER_SECOND.set(ev.metrics.llm_tokens_per_second)
    STT_AUDIO_DURATION.set(ev.metrics.stt_audio_duration)
    TTS_TTFB.set(ev.metrics.tts_ttfb)

3. Configure Prometheus: To make sure Prometheus picks up your metrics, add the endpoint to your

prometheus.yml

configuration like this: `scrape_configs: - job_name: 'livekit' scrape_interval: 10s static_configs: - targets: ['localhost:8000']`Then, just restart Prometheus so these changes take effect. 4. Visualize in Grafana: Finally, add Prometheus as a data source in Grafana. From there, you can create dashboards to easily visualize metrics such as LLM token throughput, STT audio duration, and TTS latency. If you have any questions or need a hand with any of these steps, please don’t hesitate to ask. I’m here to help!