Does livekit tests and evals push data to the cloud?

Thanks for reaching out, Amal. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I am talking about agent tests written using pytest or vitest

LiveKit Agents automatically upload observability data at session end — this includes telemetry, transcripts, audio recordings, and related logs used for debugging and analysis.

this doesn't contain tests or evals

Thanks — I’m responding to your question about pytest/vitest agent tests and whether they push data to LiveKit Cloud. Short answer: No—unit/integration tests run with

AgentSession

in pytest or Vitest do not automatically upload observability data to LiveKit Cloud unless the test explicitly connects the agent to a LiveKit room in a project with Agent Observability enabled. Key points: • Local tests using

AgentSession

are text-only by default and are designed for isolated behavioral validation; they don’t generate audio recordings or full observability telemetry. • Observability uploads occur for sessions that are actually connected to a LiveKit room and the project has Agent Observability enabled; only those connected sessions upload telemetry, transcripts, audio, and logs at session end. • If your tests call

session.connect()

or otherwise join a real LiveKit room in a cloud project, that could trigger upload—otherwise test runs stay local. If you want, tell me: • Are your tests calling

session.connect()

or joining a room URL? • Do you have Agent Observability enabled in your project? I can then recommend ways to ensure tests remain local or to safely run connected tests without uploading unwanted data. Sources: Why is my Agent insights data missing? | LiveKit | Testing and evaluation | LiveKit Documentation | Images | LiveKit Documentation | Where to find your Agent Logs | LiveKit, with interactive elements

I got this from the blog https://livekit.com/blog/streamline-troubleshooting-with-agent-observability

Automated testing, evaluations, and simulations built on top of session data, so you can continuously validate changes to prompts, models, and tools before rolling them out to production traffic.

is it available on the platform?

Thanks — I’m responding to your question about the blog line saying automated testing/evaluations/simulations are built on session data and whether that capability is available on the platform. Short answer: Not yet. That automated testing/evaluation/simulation feature is a planned future capability and is not currently available on LiveKit. What is available now: • Agent Observability provides session-level insights (synchronized audio, transcripts, traces, logs) that enable troubleshooting. • You can write unit/integration tests for agents locally (pytest/Vitest) and use third‑party simulation tools (examples: Bluejay, Hamming, Roark) for larger-scale simulations today. If you’d like, tell me whether you want to: • Keep tests fully local (I can show how to avoid uploads), or • Run connected simulations now (I can outline how to safely connect tests to a room and what triggers observability uploads). Sources: Streamline troubleshooting with Agent Observability | LiveKit | Streamline troubleshooting with Agent Observability | LiveKit | LiveKit's Series C: Towards the voice-driven era of computing | LiveKit | LiveKit's Series C: Towards the voice-driven era of computing | LiveKit, with interactive elements