How do I use an agent if my room has e2ee enabled

Thanks for reaching out, Andres. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I am using python

Thanks — you're using Python. I'll walk you through what you need and the practical options. Short answer • An agent can join an e2ee room, but agents do not decrypt E2EE media by default. To have the agent process audio/video you must explicitly enable e2ee for the agent connection and ensure the agent has the decryption keys (or use a non‑E2EE path). What you can do (options) • Join the room with E2EE enabled (agent participates as a normal participant): ◦ Use

ctx.connect(e2ee_enabled=True)

and start your

AgentSession

as usual. ◦ The agent will receive encrypted frames; it only decrypts if it has the keys. • Give the agent access to keys (advanced/custom): ◦ Provide the agent the same key material used by participants (via your key management). Only then can the agent decrypt frames locally. ◦ This requires implementing a key exchange/key provider so the agent is included in the E2EE trust circle. • Workarounds if you cannot / do not want to include the agent in the E2EE trust: ◦ Use a separate non‑E2EE backend room for the agent and bridge media between rooms. ◦ Disable E2EE for rooms where agent processing is required. ◦ Selective encryption patterns (encrypt only user-to-user while allowing agent access) — requires custom setup. Example (Python sketch)

# inside your JobContext entrypoint
await ctx.connect(e2ee_enabled=True)

session = AgentSession(
    stt=..., llm=..., tts=...,
    room_input_options=RoomInputOptions(participant_kinds=["STANDARD","AGENT"])
)

await session.start(agent=MyAgent(), room=ctx.room)
# Note: agent can only decode encrypted media if it has the keys

Important limitations • Agents generally cannot decrypt E2EE streams unless explicitly included in key negotiation or given keys via a custom provider. • LiveKit server/SFU cannot read plaintext when E2EE is enabled. If you’d like, tell me: • How you currently distribute E2EE keys (KMS / manual / other), and • Whether you prefer the agent to decrypt media or to use a bridged non‑E2EE room. I’ll then give exact Python snippets for either (A) including the agent in key exchange, or (B) a bridged non‑E2EE pattern. Sources: Agent session | LiveKit docs | Job lifecycle | LiveKit docs | Agent server options | LiveKit docs

so if I connect with e2e_options the

_create_video_stream

method will have decrypted video?

Short answer: No —

_create_video_stream

will receive encrypted frames unless the agent has the decryption keys (or is included in the E2EE key exchange). Simply connecting with `e2e_options`/`e2ee_enabled=True` does not automatically give the agent plaintext video. What to do so

_create_video_stream

gets decrypted video: • Include the agent in the E2EE trust circle by providing it the same key material used by participants (implement a key provider / key exchange so the agent receives keys). • Or run the agent in an environment that performs the insertable-stream/Encoded Transform decryption using those keys (WebRTC stack or equivalent crypto must support it). If you cannot give the agent keys, alternatives: • Bridge to a non‑E2EE room where the agent can see plaintext media. • Disable E2EE for rooms that require agent processing. • Use selective encryption patterns (custom) so agent-accessible tracks remain decryptable. If you want, tell me: • how you currently manage E2EE keys (KMS/manual/other), and • whether you prefer to include the agent in key negotiation or use a bridged non‑E2EE approach. I’ll provide exact Python code for either including the agent in key exchange or for a bridging pattern.