Thanks — you're asking how to avoid the ~2s warm‑up each time control is handed between A and B in Python, and whether caching or another approach can eliminate that latency. I’ll explain practical options you can try and a few diagnostic questions so I can suggest the best next step. Short answer • Prewarm heavy components (models, VAD, TTS, LLM clients) once per worker process and keep them in a shared place like

proc.userdata

. That removes repeated initialization for sessions that run in the same worker process. • If A and B run in different processes, you’ll need either a shared-process approach or an external service/caching layer (or IPC) so both agents reuse the same preloaded resources. What you can do now 1) Prewarm in the worker process • Use the AgentServer

setup_fnc

/

prewarm

to load expensive resources once (e.g.,

silero.VAD.load()

) and store them in

proc.userdata["vad"]

or similarly named keys. • When a new session/job starts in that same process, read from

proc.userdata

instead of reloading. This removes the warm-up cost for agents running in the same process. 2) Run both agents in the same worker process (if feasible) • If A and B can be managed inside one worker process, they can both access the same prewarmed instances directly, avoiding handoff reinitialization. 3) External caching / shared service (when agents run in different processes) • Host the heavy model or runtime in a small local service (microservice) or in-memory cache that processes call into (RPC/REST/GRPC). Agents request inference/operations from that service — they avoid local per-session initialization. • Alternatively, use an IPC mechanism (shared memory, Unix domain socket, or a lightweight local server) to share loaded models between processes. 4) Reuse client connections/resources • For services with connection overhead (LLM APIs, TTS servers), reuse persistent client instances rather than re-creating clients on each handoff. Diagnostics / questions to narrow recommendations • How are A and B deployed? Are they guaranteed to run in the same worker process or are they separate processes/containers? • What specifically is being “warmed up”? (Examples: Silero VAD model load, LLM client handshake, TTS engine initialization, loading large model files) • Is the ~2s measured during model loading, network connection, or some other step? If you can share a short trace/log around the handoff, that helps pinpoint the bottleneck. • Do you control the runtime environment so you can run a persistent helper service or change the process model? If you tell me: • whether A and B share a process, and • which component(s) are causing the delay, I can give a concrete Python example showing how to prewarm into

proc.userdata

, or show a minimal local microservice pattern (RPC server + client) to share preloaded models between agents. Sources: Gemini Realtime Agent with Live Vision | LiveKit Documentation | Self-hosted deployments | LiveKit Documentation | VAD Metrics | LiveKit Documentation | Silero VAD plugin | LiveKit Documentation | MCP Agent | LiveKit Documentation, with interactive elements