Does your Realtime models support inference as well? I see that STT, LLM, and TTS models support LiveKit Cloud inference for OpenAI and Gemini, but I couldn’t find any info about the Realtime feature for speech-to-speech or duplex with OpenAI or Gemini in the docs: https://docs.livekit.io/agents/models/realtime/

Thanks for reaching out, Abdul. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

l did tried this but getting this error: site-packages/livekit/plugins/openai/realtim e/realtime_model.py", line 324, in init raise ValueError( ValueError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

Thanks — I’m responding to the error you shared about the RealtimeModel complaining that the OpenAI API key is missing. What’s happening • The OpenAI realtime plugin requires an API key when you initialize

openai.realtime.RealtimeModel

. The plugin checks for either an

api_key

parameter passed to the constructor or the

OPENAI_API_KEY

environment variable. Your error shows neither is present, so the constructor raises that ValueError. How to fix it (two options) 1) Pass the key when creating the RealtimeModel • Add

api_key

to the constructor:

from livekit.plugins import openai
import os

realtime = openai.realtime.RealtimeModel(
    model=realtime_model_id,
    voice=realtime_voice,
    api_key=os.getenv("OPENAI_API_KEY")  # or put the literal key here (not recommended)
)

• Using

os.getenv("OPENAI_API_KEY")

keeps the same source but explicitly passes it to the client. 2) Set the OPENAI_API_KEY environment variable • Set the environment variable in the shell, CI, or your process manager before running the app:

export OPENAI_API_KEY="sk-..."
# on Windows Powershell:
$env:OPENAI_API_KEY="sk-..."

• Or add it to your .env and load it in development. Extra checks • Make sure the key you provide is valid for the OpenAI Realtime API. • If you’re testing with Gemini (the gemini branch), the equivalent provider expects Google credentials (e.g.,

GOOGLE_API_KEY

or other Google auth), so ensure the provider-specific env var or constructor arg is set for that branch. Plugin installation • Confirm the OpenAI plugin is installed (if you haven’t already):

pip install "livekit-agents[openai]"

If you’d like, I can: • Show a minimal end-to-end example that includes environment loading (e.g., using python-dotenv). • Help you choose whether to pass the key in code or rely on environment variables for your deployment. Which option would you prefer or do you want the example with dotenv? Sources: OpenAI Realtime API plugin guide | LiveKit Documentation | Gemini Live API plugin | LiveKit Documentation

But my issue was to confirm if realtime models of openai and gemini can go through livekit cloud inference gateway without providing our OpenAI or Gemini keys ? it works fine with pipeline STT, LLM and TTS inference but there isn't option for realtime inference

Thanks — I’m responding to your question about whether OpenAI and Gemini Realtime models can be used through the LiveKit Cloud inference gateway without supplying your own provider keys. Short answer: No — Realtime models require your provider credentials. What’s happening • LiveKit Inference (the managed gateway) lets you call STT, LLM, and TTS models using only your LiveKit credentials; you do not need to provide the underlying provider API keys for those pipeline models. • Realtime models (OpenAI Realtime, Google Gemini Live, xAI Grok Voice, etc.) are not available through that managed, keyless inference path. They must be used via the LiveKit realtime plugins (e.g.,

openai.realtime.RealtimeModel

,

google.realtime.RealtimeModel

) and those plugins require that you supply the provider’s API key or credentials. Why this matters • Realtime models stream raw audio in/out over WebSockets and need direct provider authentication, so LiveKit requires you to provide the provider-specific API key (for example

OPENAI_API_KEY

for OpenAI Realtime or Google credentials for Gemini). • That’s why your code raised the ValueError — the OpenAI realtime plugin expects either the

api_key

argument or the

OPENAI_API_KEY

environment variable. If you want to avoid managing provider keys • Currently there is no option to use OpenAI/Gemini Realtime models through the LiveKit Cloud inference gateway without your provider keys. • If you prefer a keyless flow, you can continue using the non-realtime pipeline (STT → LLM → TTS) via LiveKit Inference, which does not require provider-specific keys. Next steps I can help with • Show how to securely provide the provider key (env var or constructor) for your Realtime plugin. • Provide a short example comparing latency/complexity between Realtime vs STT+LLM+TTS to help choose the right path. Which would be most helpful? Sources: Models overview | LiveKit Documentation | Realtime models overview | LiveKit Documentation | Google Gemini LLM | LiveKit Documentation | OpenAI Realtime API plugin guide | LiveKit Documentation | Gemini Live API plugin | LiveKit Documentation