I have an issue where OTEL metrics in Grafana are not showing the STT metrics data.

Thanks for reaching out, Alejandro. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

STT metrics are being emitted and I see them on the logs related to the metrics_collected session event. All the other metrics are being shown by Grafana, its only STT metrics the one that is missing

Thanks — that’s helpful and narrows the problem. Since STT metrics are emitted and other metrics show up in Grafana, the issue is likely in one of these areas: metric naming/labels, collector processing (filtering/translation), or Grafana query/datasource mapping specifically for STT metrics. I’ll walk through targeted checks and then ask for a couple quick config/log snippets so I can pinpoint the cause. Targeted checks (quick to run) • Verify exact metric names and labels from the logs: ◦ Copy one full logged

metrics_collected

event for STT (JSON or full log line). I need the metric name(s) and labels as emitted. • Check for metric renaming or filtering in the OTEL Collector: ◦ Look for processors like

metricsfilter

,

transform

or

batch

that could drop or rename STT metrics. ◦ If you use a translation or attribute-based filter, ensure STT metric names/labels match the filter rules. • Confirm exporter/mapping to your Grafana data source: ◦ If you export to Prometheus, ensure the collector’s Prometheus exporter is exposing the STT metrics endpoint and that Prometheus scrape config includes that target. ◦ If you export via OTLP directly to Grafana Cloud or a hosted metrics backend, confirm the exporter includes the STT metrics stream and no resource/namespace exclusions apply. • Validate metric type compatibility: ◦ Check if STT metrics use a type Grafana or your backend treats differently (e.g., exemplars, histograms, summaries) that require different query handling. • Time-range / label filtering in the dashboard: ◦ On a raw metrics explore panel, search for the metric name without any label filters. Remove dashboard-wide filters that might exclude STT-specific labels (e.g., service name, environment). • Ensure no schema collisions: ◦ If another metric uses the same base name but different labels/type, the backend might merge or hide one variant. Compare STT metric name + label set against other metrics’ names. Please provide these 3 items (the most helpful info) 1. One example full log line or JSON of the STT

metrics_collected

event showing metric names and labels (remove any secrets). 2. Your OTEL collector pipeline (or the agent’s exporter config) — the part that receives from the Agent and exports to your backend. 3. Which Grafana data source you use for these metrics (Prometheus scrape, Grafana Cloud, or direct OTLP), and an example Grafana query you used (if any). What I’ll do once you send them • Inspect the emitted metric name/labels for naming mismatches. • Check the collector pipeline for filters/renames that would affect STT metrics and suggest exact yaml fixes. • Suggest the correct Grafana query or scrape configuration to make the STT metrics visible. If you prefer, start by pasting just the full

metrics_collected

log line and the collector’s receiver/exporter sections — that’s often enough to find the mismatch.

This is an example of a STT metrics log entry

2026-03-16T13:00:17.491454147Z {"timestamp": "2026-03-16T13:00:17.490858+00:00", "level": "DEBUG", "logger": "agent.metrics", "message": "Metric entry - type='stt_metrics
' label='livekit.plugins.openai.stt.STT' request_id='' timestamp=1773666017.490154 duration=1.1069130413234234 audio_duration=2.836 streamed=False metadata=Metadata(model
_name='whisper-1', model_provider='<http://api.openai.com|api.openai.com>')", "pid": 540, "job_id": "AJ_ZbjSnWnz6gJF", "call_id": "e60fc986-9bda-123f-a6ac-fea681b75b24", "trace_id": "935217b21a1
5001b1e96942aada03dce", "span_id": "e14505ac051da0b5"}

Thanks — that log is helpful and confirms STT metrics are being emitted. I’ll focus on why those emitted metrics aren’t showing in Grafana even though other metrics are. Quick observations from the log you shared • The metric is emitted with type/name

stt_metrics

and label

livekit.plugins.openai.stt.STT

. • The payload includes fields such as

duration

,

audio_duration

,

streamed

, and

metadata

(e.g.,

model_name

,

model_provider

). • This shows the Agent is correctly producing STT metric events, so the problem is in the pipeline between emission and Grafana (collector filtering/mapping, exporter, or Grafana query/mapping). Next targeted checks — please run these or share the outputs 1. Search raw metrics in your backend/Explore ◦ In Grafana Explore (or directly in Prometheus if you use it), query for the metric base name without filters: search for

stt_metrics

(or partial matches like

stt_*

) to see what variants exist. ◦ If you use Prometheus, try a label-less query like:

stt_metrics

and also

count(stt_metrics)

over a recent time window. 1. Provide your OTEL Collector / Agent exporter config ◦ Share the receiver -> processors -> exporters pipeline (remove secrets). I specifically need to see any

metricsfilter

,

transform

,

metricstransform

or relabeling rules that might drop/rename STT metrics. 1. Share how you export metrics to Grafana ◦ Which data source are you using in Grafana? (Prometheus scrape, Grafana Cloud via OTLP, direct OTLP, Grafana Agent, etc.) ◦ If Prometheus, share the scrape config that targets the collector’s Prometheus exporter. ◦ If OTLP -> Grafana Cloud, share the collector exporter config (endpoint, resource attributes settings). 1. Paste an example of the STT metric as exposed by the collector (if available) ◦ If you can curl the collector/exporter endpoint and show one STT metric line (Prometheus text exposition format) that’s ideal. Example: the scrape endpoint output or an OTLP-exported metric sample. Common misconfigurations to check now (quick actionable pointers) • Metric name normalization: Some collectors or exporters rename metrics (e.g., replace

.

with

_

or add prefixes). Confirm

stt_metrics

is not being renamed to something else like

livekit_stt_metrics

. • Metrics filtering: Look for processors that exclude metrics by name/pattern. STT metrics may be filtered out by a pattern that assumed only standard names. • Label types and metadata: If STT metrics include complex fields (e.g.,

metadata

struct), the collector might drop them or fail to convert the metric. Check for processor errors in collector logs. • Metric type handling: If STT metrics are emitted as a different metric type (e.g., summary/histogram) the dashboard queries might not surface them without correct aggregation functions. • Dashboard-level filters: Ensure dashboard or panel-level variables/label filters aren’t excluding STT-specific labels like

service=...

or

plugin=...

. What I can do once you share the items above • Inspect the collector pipeline and point out exact changes to prevent STT metrics being dropped or renamed. • Provide the exact Grafana query to use once I confirm the final metric name/labels. • Offer a minimal collector yaml snippet (receivers/processors/exporters) that preserves STT metrics to your backend. If you want to start small, paste: • The collector’s processors and exporters section (or the agent exporter config), and • Which Grafana data source you’re using (Prometheus or OTLP), plus a quick result of searching

stt_metrics

in Grafana Explore. I’ll review those and give precise changes you can apply.

this is my current OTEL configuration on the agent def setup_telemetry(): from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor settings = get_settings() public_key = settings.traces_otel_public_key secret_key = settings.traces_otel_secret_key host = settings.traces_otel_host if secret_key and public_key: grafana_auth = base64.b64encode(f"{public_key}:{secret_key}".encode()).decode() else: grafana_auth = None os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = host if grafana_auth: os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"Authorization=Basic {grafana_auth}" trace_provider = TracerProvider() trace_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) set_tracer_provider(trace_provider)

Thanks — this is very helpful. You’ve shown the Agent is configured to export traces via OTLP/HTTP to Grafana, but that snippet only sets up tracing (spans). Since STT metrics are emitted by the Agent as metric events, we need to confirm how metrics are being exported from the Agent to your collector/backend (metrics exporters are separate from tracing exporters). What to check next (quick summary) • Confirm the Agent is configured to export metrics (not just traces) via OpenTelemetry. The snippet you shared configures a

TracerProvider

and an OTLP span exporter — it does not show a metric exporter or metrics SDK setup. Metrics will not be sent to OTLP unless a metrics exporter is also configured. • If you rely on a local OTEL Collector, confirm the Agent is sending metrics to the collector and that the collector pipeline is not filtering/dropping STT metrics. • If another path is used (Prometheus exposition, Grafana Agent, or direct OTLP metrics to Grafana Cloud), confirm that metrics-specific exporter/receiver config is present. What I need from you to give exact changes • Do you have a metrics exporter configured in the Agent code or are you relying on an OTEL Collector to scrape/receive metrics? If yes, paste the metrics exporter or agent metrics setup (remove secrets). • If you use an OTEL Collector, paste the collector pipeline config (receivers, processors, exporters), especially any

metrics

receiver and any processors like

metricsfilter

,

metricstransform

,

transform

, or relabeling rules. • Which Grafana data source are you using for metrics? (Prometheus scrape of a collector endpoint, OTLP directly to Grafana Cloud, Grafana Agent, etc.) Immediate actionable suggestions you can try now 1. If metrics exporter is missing in the Agent: ◦ Add a metrics SDK + OTLP metrics exporter so the Agent sends metrics. The tracing-only setup won’t export metric events. 1. If you already have a collector: ◦ Check collector logs for errors related to STT metrics conversion. Complex attributes (like the

metadata

struct in your log) can cause processors to drop metrics if conversion fails. ◦ Search the collector config for any filters that might match

stt_metrics

(name patterns, service/resource exclusions) and temporarily disable them to test. 1. In Grafana Explore: ◦ Search for the raw metric name

stt_metrics

(no dashboard filters). If you see no results, it confirms metrics never reach the backend; if you see variants, note the exact name/labels and share them. 1. If your backend is Prometheus: ◦ Confirm the Prometheus scrape target includes the collector’s Prometheus exporter endpoint and that

stt_metrics

shows on the exporter scrape page. If you want, start by pasting either: • The Agent code that sets up metrics export (metrics SDK/exporter), or • The OTEL Collector pipeline yaml (receivers/processors/exporters). Once I see one of those, I’ll provide the exact code/config changes to ensure STT metrics get exported and appear in Grafana, and I’ll give the correct Grafana query to surface them.

how I do that

Thanks — I’ll walk you through exactly how to enable and export STT metrics from the agent (you currently only have tracing set up), and how to verify they reach Grafana. I’ll keep this step‑by‑step and include minimal example code and quick checks you can run. Summary of the problem I’m addressing • Your agent logs show STT events (metrics_collected) are emitted. • Your current agent code only configures tracing (TracerProvider + OTLPSpanExporter) and does not configure an OTEL metrics pipeline, so STT metrics aren’t being exported. What you need to do (high level) 1. Configure an OpenTelemetry metrics provider in the agent and add a metrics exporter (OTLP or Prometheus) so metrics are sent out. 2. Ensure the collector (if you use one) receives those metrics and does not filter/rename them. 3. Verify the metrics reach your Grafana data source (Prometheus/OTLP) and adjust the Grafana query if needed. Step-by-step: enable metrics export in the agent (Python) 1) Install required OTEL metric packages (if not installed) • Ensure you have the OTEL metric exporter and SDK packages available for your environment. 2) Add a MeterProvider and OTLP metrics exporter (example) • This is the minimal pattern to create an OTLP metric exporter and a periodic reader so metrics are exported to the collector or Grafana endpoint.

from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
import os

# point to your collector / Grafana OTLP endpoint
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<http://your-collector:4317>"
# if you need headers, set OTEL_EXPORTER_OTLP_HEADERS similarly (omit secrets here)

exporter = OTLPMetricExporter(endpoint=os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"])
reader = PeriodicExportingMetricReader(exporter, export_interval_millis=5000)
provider = MeterProvider(metric_readers=[reader])
metrics.set_meter_provider(provider)
meter = metrics.get_meter(__name__)

3) Expose STT metrics from the agent into the meter • The agent emits STT events. Use either observable instruments or update counters/gauges when the

metrics_collected

event fires. Example pattern using a synchronous instrument (update on event):

stt_duration_gauge = meter.create_observable_gauge("stt_duration_seconds")
# If your SDK doesn’t support observable easily, consider using an updatable counter/measure
# Alternatively, on each metrics_collected event, call a synchronous instrument update

# Pseudocode inside your event handler:
def on_stt_metrics(metrics):
    # extract numeric fields from metrics (duration, audio_duration, ...)
    duration_value = metrics.duration
    # update an instrument (example API varies by OTEL python versions)
    # For many SDKs you can use meter.create_counter or meter.create_observable_gauge callbacks
    some_updatable_instrument.record(duration_value, attributes={"plugin": "openai", "model": metrics.metadata.model_name})

Notes: • The exact OTEL Python metric API depends on your installed OTEL packages and version. The core idea is to create instruments and record values when STT events arrive. • If you prefer sending metrics to a local OTEL Collector, set endpoint to the collector’s OTLP receiver (default gRPC 4317 or HTTP 4318). Collector side checks (if you use an OTEL Collector) • Verify the collector has an OTLP receiver enabled and a metrics pipeline: ◦ receivers: otlp (grpc/http) ◦ service.pipelines.metrics: receivers: [otlp], processors: [...], exporters: [prometheus, otlp, your_backend] • Look for processors that might filter or rename metrics:

metricsfilter

,

metricstransform

,

transform

. Temporarily disable these or ensure they allow

stt_metrics

/

stt_duration

names. • If the collector exports to Prometheus, ensure the Prometheus exporter endpoint is reachable and scrapeable. Quick commands to test where metrics are lost 1. Agent -> Collector • Enable debug logs on the agent’s OTEL metric exporter and confirm it reports successful exports or errors. 1. Collector -> Prometheus endpoint • Curl the collector Prometheus exporter endpoint, e.g.: curl http//&lt;collector host&gt;<prometheus-exporter-port>/metrics | grep stt • If you see lines containing your STT metric names there, the agent->collector path is working. 1. Grafana -> Prometheus • In Grafana Explore (or Prometheus UI), search for the exact metric name you created (try

stt_duration_seconds

,

stt_duration

, or

stt_metrics

). If you don’t know exact name, search for

stt_

prefix or

livekit

labels. Common pitfalls and how to fix them • Only traces configured: the TracerProvider you set does not export metrics — you must configure MeterProvider and metric exporter separately. • Metric name mismatch: the collector or exporter may rename

stt_metrics

to another name; inspect the collector’s metrics output to find the exact metric name to query in Grafana. • Filters/processors dropping metrics: check collector processors for filters that exclude STT metrics and adjust them. • Complex attributes: if STT metadata is a nested object, the collector may drop attributes; flatten or only attach simple attributes (string values) to metrics. If you want, I can: • Provide a concrete metric-recording snippet tailored to the exact OTEL Python package versions you use (tell me your OTEL package versions). • Review your agent code snippet where you handle

metrics_collected

and suggest the exact place to record metrics to OTEL. • Review your OTEL Collector pipeline yaml to spot any filtering/translation that could drop STT metrics. Next step for me to give you exact code/config edits • Paste the agent code that handles

metrics_collected

(the function body) and the versions of opentelemetry packages installed, or • Paste your OTEL Collector metrics pipeline (receivers/processors/exporters) if you use a collector. Which option would you prefer? Sources: STT Metrics | LiveKit Documentation | Agents (Node.js) | livekit/livekit_composite | DeepWiki | Terms of Service | LiveKit | LLM Metrics | LiveKit Documentation | Announcing LiveKit Cloud | LiveKit

what are the spans?

Great question — thanks for asking. I’ll explain spans in simple terms, how they relate to tracing, and how that differs from metrics so you can see why your tracing setup won’t send STT metrics. What a span is • A span is a single timed operation or unit of work in a distributed system. Think of it as a labeled stopwatch around a function, request, or step (for example: “handle incoming audio”, “call STT API”, or “save transcription”). • Each span records a start time, an end time (duration), and optional metadata (attributes/tags) such as service name, status, or IDs that help identify the operation. • Spans can be nested or linked to show causal relationships (a parent span with child spans) so you can trace a request across services. How spans fit into tracing • Tracing is a collection of spans that together show the lifeline of a request or workflow across components. Traces are useful for debugging performance, latency, error propagation, and seeing the sequence of calls across services. • Spans are exported to a tracing backend (via an OTLP span exporter in your agent). That’s what your current TracerProvider + OTLPSpanExporter does. How spans differ from metrics • Purpose: ◦ Spans (traces) show detailed, per-request timing and causal flow. ◦ Metrics are aggregated numeric measurements you query over time (counters, gauges, histograms), like “requests per second,” “average STT processing duration,” or “audio processed seconds.” • Granularity: ◦ Spans are per operation and often high-cardinality (many unique traces). ◦ Metrics are aggregated and better for dashboards, alerts, and long-term monitoring. • Export path: ◦ You configure a TracerProvider (spans) separately from a MeterProvider (metrics). Having spans exported does not export metrics automatically. Why this matters for your STT metrics • Your agent is already exporting spans (traces) because you set up a TracerProvider and OTLP span exporter. • STT metrics (the

metrics_collected

events you see in logs) are numeric metrics, not spans. To get them into Grafana you must configure a metrics pipeline: create a MeterProvider in the agent or forward metrics to an OTEL Collector that receives and exports metrics, then ensure Grafana reads that metrics backend. • In short: spans explain “what happened and when”; metrics answer “how much / how many / how long over time.” You need the metrics exporter (MeterProvider/OTLPMetricExporter or Prometheus exposition) for STT metrics to appear in Grafana. If you want, I can: • Show a small example of how to record STT values as metrics in your agent (based on your OTEL Python versions). • Explain how to view spans vs metrics in Grafana so you can use traces for debugging and metrics for dashboards. Which would help you next — a short code example for metrics export, or a quick note on how to inspect spans in Grafana?

I found this github issue on the livekit project that looks similar to mine, and contains the following reply from a developer: Thanks for opening the issue! STT metrics are not added to any spans right now because of a few things: • They are reported very frequently, not just when there is a transcript, but also when there's some amount of silence (e.g. usage) • Currently, user speech spans are correlated with VAD states, not with STT transcripts, mainly for maximum compatibility since not every vendor supports accurate transcript timing or speech states. One thing I can see is that we could report something for each transcript, though not necessarily for any spans.

Thanks — that developer comment is useful context. I’ll briefly confirm what it means and give clear next steps so you can get STT metrics into Grafana. What the developer reply means (short) • STT metrics are emitted frequently (including during silence/usage), so they are not currently attached to OpenTelemetry spans. Spans represent discrete operations and attaching high-frequency STT events would create too many spans and cause compatibility/performance issues. • LiveKit correlates speech spans with VAD states, not STT transcripts, so STT events are emitted separately from traces. • STT metrics are still available — you just need to export them via a metrics pipeline (MeterProvider/OTLP or Prometheus) rather than relying on spans. Why your traces don’t include STT metrics • Your agent code configures a TracerProvider and an OTLP span exporter (so traces are being sent). • Traces do not automatically include metric events; metrics require a separate MeterProvider and metrics exporter. That’s why your STT logs show metrics emitted locally but they aren’t appearing in Grafana. Actionable next steps to get STT metrics into Grafana 1. Add metrics export in the agent (quick summary) ◦ Create a MeterProvider and an OTLPMetricExporter (or expose Prometheus) in your agent. ◦ When the STT

metrics_collected

event fires, record the numeric fields (e.g.,

duration

,

audio_duration

,

end_of_utterance_delay

,

transcription_delay

) on OTEL instruments (counters/gauges/histograms) or update observable callbacks. 1. Minimal verification flow ◦ Agent → Collector: ensure the agent’s OTLP metrics exporter is sending to your collector (or Grafana OTLP endpoint). ◦ Collector → Backend: confirm the collector’s metrics pipeline receives and forwards STT metrics (no filters). ◦ Backend → Grafana: curl the collector’s Prometheus endpoint (or query your backend) to find the STT metric names, e.g.: curl http//&lt;collector host&gt;<prometheus-port>/metrics | grep stt • Grafana: in Explore query the exact metric name (try

stt_duration

or search

stt_

). 1. Quick code sketch (what to add to your agent) ◦ Add a MeterProvider + OTLP metric exporter (or Prometheus exporter) alongside your TracerProvider. ◦ In your STT event handler, extract numeric values and record them to OTEL instruments. If you want, I can give a tailored snippet for your OTEL Python version. 1. Collector checks (common gotchas) ◦ Look for processors like

metricsfilter

,

metricstransform

, or relabeling that may drop or rename

stt_*

metrics — temporarily disable them for testing. ◦ If STT metrics include nested metadata objects, flatten attributes to simple strings to avoid conversion drops. If you want help right now, tell me one of these and I’ll provide exact code/config edits: • Your opentelemetry Python package versions (so I can give the correct metric API snippet), or • Paste the agent function that handles

metrics_collected

and I’ll show exactly where and how to record those values to OTEL, or • Paste your OTEL Collector metrics pipeline yaml and I’ll point out any filters/renames and suggest fixes. Which would you prefer? Sources: STT Metrics | LiveKit Documentation

Can you provide an example code of how to export the metrics via a metrics pipeline?

Thanks — below is a concise, practical example showing how to capture LiveKit STT

metrics_collected

events in the agent, record them to OpenTelemetry metrics, and export them via OTLP to an OTEL collector (or directly to a backend that accepts OTLP). I also include quick verification steps for Prometheus/Grafana. Notes before you start • Your current snippet configured only tracing (TracerProvider). Metrics require a separate MeterProvider and exporter. • The Python OpenTelemetry metric API can vary by version; this example uses the modern OTEL SDK pattern (MeterProvider + PeriodicExportingMetricReader + OTLPMetricExporter). If your installed OTEL package versions differ, tell me the versions and I’ll adapt the snippet. 1) Minimal dependencies (examples) • opentelemetry-api • opentelemetry-sdk • opentelemetry-exporter-otlp[grpc] (or otlp-http if you prefer) • livekit agents package 2) Example agent code (capture STT events -> OTEL metrics)

import asyncio
import os
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter

# Set OTLP endpoint (collector or Grafana OTLP endpoint)
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<http://your-collector:4317>"
# If you need headers: os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Authorization=Bearer <token>"

# 1) Setup MeterProvider + OTLP exporter
metric_exporter = OTLPMetricExporter(endpoint=os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"])
metric_reader = PeriodicExportingMetricReader(metric_exporter, export_interval_millis=5000)
provider = MeterProvider(metric_readers=[metric_reader])
metrics.set_meter_provider(provider)
meter = metrics.get_meter("livekit.stt")

# 2) Create instruments you will record when STT events arrive
# Use a histogram for duration-like metrics (captures distributions)
stt_duration_hist = meter.create_histogram(
    name="livekit_stt_duration_seconds",
    description="STT processing duration in seconds",
    unit="s"
)

stt_audio_duration_hist = meter.create_histogram(
    name="livekit_stt_audio_duration_seconds",
    description="Length of audio processed by STT in seconds",
    unit="s"
)

# Optional counter for transcript events
stt_transcript_counter = meter.create_counter(
    name="livekit_stt_transcripts_total",
    description="Number of STT transcript events"
)

# 3) Wire the LiveKit STT event to record metrics
# Replace with your actual agent/session code
def stt_wrapper(metrics_event):
    # metrics_event fields: duration, audio_duration, end_of_utterance_delay, transcription_delay, streamed, metadata...
    attrs = {
        "plugin": "openai",  # adapt from metrics_event label/metadata if available
        "model": getattr(metrics_event.metadata, "model_name", "unknown") if getattr(metrics_event, "metadata", None) else "unknown",
        "streamed": str(getattr(metrics_event, "streamed", False))
    }

    # record numeric values
    try:
        duration = float(getattr(metrics_event, "duration", 0.0))
        audio_duration = float(getattr(metrics_event, "audio_duration", 0.0))
    except Exception:
        duration = 0.0
        audio_duration = 0.0

    stt_duration_hist.record(duration, attributes=attrs)
    stt_audio_duration_hist.record(audio_duration, attributes=attrs)
    stt_transcript_counter.add(1, attributes=attrs)

    # If you want to capture other numeric fields:
    # stt_eou_delay_hist.record(getattr(metrics_event, "end_of_utterance_delay", 0.0), attributes=attrs)

# Example: registering handler with LiveKit session (pseudo-code)
# self.session.stt.on("metrics_collected", lambda me: asyncio.create_task(async_handler(me)))
def async_handler(metrics_event):
    # If your environment requires async, wrap with create_task
    stt_wrapper(metrics_event)

# In your agent initialization, ensure you register the handler appropriately:
# self.session.stt.on("metrics_collected", lambda me: asyncio.create_task(your_async_handler(me)))

3) Collector / backend considerations • Point OTLP exporter to your collector OTLP receiver (default gRPC: 4317). If you want direct Grafana OTLP ingestion, use Grafana Cloud OTLP endpoint and correct headers. • If you prefer Prometheus: ◦ Configure the collector to receive OTLP metrics and export to Prometheus (collector’s prometheus exporter). ◦ Scrape the collector’s Prometheus endpoint from Grafana (or add it as a Prometheus data source). Example OTEL Collector metrics pipeline (simple)

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
  logging:
    loglevel: debug

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheus, logging]

4) Quick verification steps • Agent -> Collector: ◦ Enable debug logs on the agent/exporter. Confirm metric export success in agent logs. • Collector -> Prometheus endpoint: ◦ curl http//&lt;collector host&gt;8889/metrics | grep livekit_stt • Grafana: ◦ Add Prometheus data source pointing at the collector Prometheus exporter (http//&lt;collector host&gt;8889). ◦ In Explore, query: livekit_stt_duration_seconds or search metrics by prefix

livekit_stt_

. 5) Troubleshooting tips • Metric names: use predictable names (avoid nested objects as attributes). Flatten metadata values (model name) to strings. • Export interval: export_interval_millis controls how often metrics are pushed; set reasonably (e.g., 5000 ms). • Cardinality: don’t attach too many high-cardinality attributes (user IDs, long IDs) to metrics, or Prometheus may struggle. • If you still don’t see metrics in Grafana: check collector logs for dropped metrics or processor filters (e.g., metricsfilter, metricstransform). If you want I can: • Tailor the code to your exact OTEL Python package versions (tell me opentelemetry-api, sdk, exporter versions). • Provide a version that uses PrometheusMetricReader directly in case you want the agent itself to expose a Prometheus endpoint (instead of pushing to OTLP). Which would you prefer: OTLP push to a collector (example above) or the agent exposing a Prometheus endpoint directly? Sources: STT Metrics | LiveKit Documentation | Agents (Node.js) | livekit/livekit_composite | DeepWiki | LiveKit&amp;#39;s Series C: Towards the voice-driven era of computing | LiveKit