I am using AgentSession and Worker options, I can see that there is one property related to Prometheus may i know what is the use of it

Thanks for reaching out, Mayank. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I want to use Prometheus setup in my project with worker options

Let's walk through setting up Prometheus with

WorkerOptions

together. Here’s how you can do it step-by-step: 1. Enable Prometheus Metrics: Start by setting the

prometheus_port

parameter in

WorkerOptions

to a valid port number (for example, 8081). This will make metrics available at

http://<worker-host>:<prometheus_port>/metrics

. Here’s a simple example in Python:

from livekit.agents import WorkerOptions, Worker

opts = WorkerOptions(
    entrypoint_fnc=entrypoint,  # Your agent's entrypoint
    prometheus_port=8081        # Enable Prometheus metrics on port 8081
)

worker = Worker(opts)
await worker.run()

2. Configure Prometheus Server: Next, add a scrape job in your

prometheus.yml

file so Prometheus knows where to find the worker’s metrics:

scrape_configs:
  - job_name: 'livekit-agent'
    static_configs:
      - targets: ['<worker-host>:8081']

3. Verify Setup: Finally, check that everything is working by visiting

http://<worker-host>:<prometheus_port>/metrics

in your browser or using

curl

. Also, take a look at the Prometheus UI under "Status > Targets" to confirm the target shows as "UP". Just a quick tip: make sure your worker’s host is accessible from the Prometheus server and that the port you picked isn’t being used by another service. If you’d like to dive deeper, the Worker Options reference has some great details. Sources: Log collection | LiveKit Docs | Worker options | LiveKit Docs | livekit-agents/livekit/agents/worker.py | tools/server/bench/prometheus.yml | Self-hosting the Ingress Service | LiveKit Docs

Ok other than Prometheus what other tools I can integrate so that I can see my logs and performance

Alongside Prometheus, you have some great options to integrate tools for logs and performance monitoring with LiveKit: • Datadog: LiveKit makes it easy to forward runtime logs to Datadog. By simply adding a Datadog client token as a secret, all your runtime logs from agent instances will be sent directly to your Datadog account. This is especially helpful since LiveKit Cloud doesn’t store runtime logs, so you can always have comprehensive access to them. • OpenTelemetry: LiveKit also comes with built-in support for OpenTelemetry, which means you can enjoy trace-based observability. You’re able to send telemetry data to any OpenTelemetry-compatible provider. This includes tracing activities like session starts, agent turns, LLM node execution, and metrics such as latency and duration. For instance, you might configure the tracer provider to work with a service like LangFuse. By using these integrations, you’ll be able to keep a close eye on agent behavior, identify any performance bottlenecks, and create custom dashboards tailored to your favorite observability tools. If you need any help setting these up or have questions, I’m here to assist! Sources: Log collection | LiveKit Docs | Logs, metrics, and telemetry | LiveKit Docs

I have self hosted my python application on ec2 then tell me integration and all

I'm glad to help you integrate Datadog and OpenTelemetry with your self-hosted Python application on EC2 for LiveKit logs and performance monitoring. Let's walk through the steps together: • Set Up Datadog for Log Collection: ◦ Install Datadog Agent: On your EC2 instance, run: `DD_API_KEY=<your-datadog-api-key> DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"`Just be sure to replace

<your-datadog-api-key>

with your actual Datadog API key. ◦ Enable Log Collection: Open

/etc/datadog-agent/datadog.yaml

and make sure logs are enabled by adding:

logs_enabled: true

◦ Configure Log Sources: Create the file

/etc/datadog-agent/conf.d/python.d/conf.yaml

to specify where your app logs are located:

logs:
  - type: file
    path: /var/log/your-python-app/*.log
    service: livekit-agent
    source: python
    tags:
      - env:production
      - role:voice-agent

◦ Restart Datadog Agent: To apply changes, run:

sudo service datadog-agent restart

◦ Log to File in Python: Make sure your app writes logs like this:

import logging
import logging.handlers

logger = logging.getLogger("livekit")
logger.setLevel(<http://logging.INFO|logging.INFO>)
handler = logging.handlers.RotatingFileHandler("/var/log/your-python-app/app.log", maxBytes=10485760, backupCount=5)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

• Integrate OpenTelemetry for Performance Monitoring: ◦ Install OpenTelemetry: You can do this by running:

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http requests

◦ Configure OpenTelemetry: Add the following to your app to set it up:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from livekit.agents.telemetry import set_tracer_provider

def configure_telemetry():
    os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<https://api.datadoghq.com/api/v2/otlp>"
    os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"DD-API-KEY={os.environ['DD_API_KEY']}"
    provider = TracerProvider()
    span_processor = BatchSpanProcessor(OTLPSpanExporter())
    provider.add_span_processor(span_processor)
    set_tracer_provider(provider, metadata={"service.name": "livekit-agent"})

◦ Call Telemetry Setup: Just remember to call

configure_telemetry()

early in your app’s entrypoint to get everything running smoothly. • Validate in Datadog: ◦ Take a moment to check your logs in Datadog Logs. ◦ Also, you can verify traces in Datadog APM. If you have any questions or want to dive deeper into custom spans or other configurations, just let me know—I’m here to help! Sources: Logs, metrics, and telemetry | LiveKit Docs | Log collection | LiveKit Docs | pnpm-lock.yaml | livekit-agents/livekit/agents/telemetry/trace_types.py

Ok I want send some fillers when my llm is making api calling

To help you send fillers while your LLM is making API calls, here are some friendly approaches you can try: • System Prompt Instructions: You can add instructions in the system prompt so the LLM naturally lets users know it’s retrieving information. For example:

agent = VoicePipelineAgent(\n    chat_ctx=llm.ChatContext().append(\n        role=\"system\",\n        text=(\n            \"You are a voice assistant. If you need to perform a function call, always tell the user that you are looking up the answer.\"\n        ),\n    ),\n)

• Static Text Messages: Another simple way is to use predefined messages like "Let me look that up..." right before making the API call:

thinking_messages = [\n    \"Let me look that up...\",\n    \"One moment while I check...\"\n]\n\nawait agent.say(random.choice(thinking_messages))

• Dynamic LLM Responses: You could also use a lightweight LLM to generate a quick filler response dynamically while the main LLM processes the request:

async def on_user_turn_completed(turn_ctx, new_message):\n    fast_llm_ctx = turn_ctx.copy(exclude_instructions=True).truncate(max_items=3)\n    fast_llm_ctx.items.insert(0, llm.ChatMessage(\n        role=\"system\",\n        content=\"Generate a short instant response like 'let me think about that'.\"\n    ))\n    async for chunk in self._fast_llm.chat(chat_ctx=fast_llm_ctx).to_str_iterable():\n        yield chunk

• Play Audio Files: Or, if it fits your use case, playing a short audio clip while waiting can be really engaging:

await play_wav_once(\"let_me_check_that.wav\", ctx.room)

These approaches can really help keep your users informed and create a smoother, more natural conversation during those moments when the system is working behind the scenes. If you'd like, I’m here to help you implement any of these—just let me know! Sources: meditation_assistant/README.mdoc | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py