is it possible to use an observer LLM without going through the inference gateway?

Thanks for reaching out, Vignesh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

yes, code example

Below is a concise example showing how to implement an observer LLM using the OpenAI plugin (no LiveKit Inference Gateway). This demonstrates: 1) creating a plugin-based LLM, 2) wiring it as an observer to receive events/messages, and 3) making a simple call. Adjust imports to match your SDK language and initialize with your provider API key.

# Python-like pseudocode — adjust to your SDK/environment

# 1) Create a plugin-based LLM (OpenAI) with your provider key
from livekit_agents import plugins, Observer, RoomClient

openai_plugin = plugins.openai.OpenAILLM(api_key="OPENAI_API_KEY")  # or use environment var

# 2) Create an observer which uses the plugin LLM
class MyObserver(Observer):
    def __init__(self, llm):
        self.llm = llm

    async def on_message(self, message):
        # Prepare prompt from observed message
        prompt = f"Summarize this message for moderation: {message.text}"

        # Call the plugin LLM directly (no inference gateway)
        response = await self.llm.complete(prompt=prompt, max_tokens=100)

        # Handle response (e.g., log, raise alert, or store)
        print("Observer LLM response:", response.text)

# 3) Attach observer to a room/client to receive events
room = RoomClient(api_key="LIVEKIT_API_KEY", room="example-room")
observer = MyObserver(openai_plugin)

# Hook observer into room event stream (pseudo-code; use actual SDK event hooks)
room.on("message", observer.on_message)

# 4) Run event loop / client
await room.connect()
await room.listen_forever()

Notes: • Replace method names (

complete

,

response.text

, event names) with the exact calls from your SDK/plugin implementation. • You are responsible for provider credentials, billing, and rate limits when using plugin LLMs. • If you want, I can provide the same example in TypeScript/Node or adapt it to your exact SDK version. Which language are you using?, with interactive elements