I want to understand how prompt caching works. I though every turn is a new prompt sent separately, with all the chat history. Or there is a cache key assigned to the first prompt request, and appended to the next calls? or it's one "session" in openai? Can I have prompt caching cross calls, using openai prompt_cache_key?

Thanks for reaching out, Alexandru. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I'm using the livekit Openai plugin.

Thanks — that helps. I’m responding to your question about whether prompt caching (using a

prompt_cache_key

) works across calls when you’re using the LiveKit OpenAI plugin. Short answer: I don’t have documentation that LiveKit’s OpenAI plugin implements or exposes a

prompt_cache_key

prompt-caching feature for chat requests, nor is there documentation showing OpenAI-side prompt caching being surfaced through the plugin. LiveKit docs do show OpenAI integrations for LLM, Realtime, STT, and TTS, and they include a TTS-specific cache example for audio reuse, but nothing about caching prompts or a

prompt_cache_key

for LLM/chat requests. To give you practical next steps and a concrete recommendation, I need one quick clarification from you: • Are you trying to reuse a key you control (your own cache layer) across separate HTTP calls/processes, or are you asking whether the LiveKit plugin / OpenAI provider itself supports a built-in

prompt_cache_key

parameter that persists across calls? Based on which of those you mean, here’s what I can help with next: • If you mean a built-in plugin/parameter (plugin/OpenAI supports

prompt_cache_key

): I’ll confirm that the plugin does not document such a feature and explain alternatives LiveKit supports (for example, re-sending chat history each turn or using the Realtime API session behaviors) and how to architect caching outside the plugin. • If you mean your own cross-call caching (you want to implement prompt caching yourself): I can outline a simple, safe pattern to cache prompt+response pairs (or embeddings) on your side and reuse them across calls to reduce tokens — including what to cache, cache-key strategies, cache invalidation ideas, and where to insert that into the LiveKit plugin call flow. Tell me which of the two you mean (built-in

prompt_cache_key

vs. your own cross-call cache), and if you want the self-managed approach, paste a short example of how you currently send turns (no keys or secrets). I’ll then give step-by-step guidance tailored to your workflow. Sources: OpenAI Realtime API plugin guide | LiveKit Documentation | OpenAI LLM models | LiveKit Documentation | OpenAI STT plugin guide | LiveKit Documentation | OpenAI TTS plugin guide | LiveKit Documentation | Agent speech and audio | LiveKit Documentation, with interactive elements

Just tell me, how prompt caching in a call session works

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!