# Configure voice processing pipeline with dataclass session = AgentSession[HelperSessionInfo]( userdata=HelperSessionInfo( phone_number=phone_number, twilio_number=twilio_number, start_time=datetime.now() ), stt=deepgram.STT(model="nova-3", language="multi"), llm=groq.LLM( model="moonshotai/kimi-k2-instruct-0905", temperature=0.1 ), tts=deepgram.TTS(model="aura-2-delia-en"), vad=silero.VAD.load(), turn_detection="vad", max_endpointing_delay=6.0, min_endpointing_delay=0.9 ) llm=groq.LLM( model="moonshotai/kimi-k2-instruct-0905", temperature=0.1 ), how can one use a custom llm here

Thanks for reaching out, Billy. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

open ai integrates with many llms, i can use any as long as they use the openai import?

Thank you for reaching out! I’d love to help clarify how the OpenAI plugin in LiveKit works. It currently supports specific LLMs that have been thoughtfully implemented with dedicated methods like

with_groq()

or

with_cerebras()

. While these LLMs do use OpenAI-compatible APIs, the integration is designed for those predefined in the plugin to ensure everything runs smoothly. Right now, supported LLMs include Azure OpenAI, Cerebras, Groq, Perplexity, and a few others. Each one has its own hardcoded base URL and authentication setup, so unfortunately, arbitrary OpenAI-compatible LLMs can’t be used unless they’re explicitly added to the plugin. If you have a particular LLM in mind, I’m here to help! I can check if it’s supported or guide you through extending the plugin for a custom integration. Just let me know how you’d like to move forward—I’m happy to assist! Sources: LiveKit integration guides | LiveKit Docs | OpenAI compatible LLMs | LiveKit Docs | OpenAI compatible LLMs | LiveKit Docs | plugins/openai/src/llm.ts | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/__init__.py

groq uses moonshot but i want to use moonshot directly

Thanks for reaching out! It looks like Moonshot LLM isn’t directly supported in LiveKit’s current plugins. Could you please let me know if Moonshot offers an OpenAI-compatible API or if you have any specific integration needs? That way, I can better understand your situation and help find some possible workarounds or custom solutions just for you.

def get_openai_client() -> AsyncOpenAI: return AsyncOpenAI( api_key=os.getenv("MOONSHOT_API_KEY"), base_url="https://api.moonshot.ai/v1" ) from openai import AsyncOpenAI moonshotai uses openai imports so i was hoping i could use it for the llm

Thank you so much for sharing those details! I want to make sure we get this right together. While MoonshotAI does use OpenAI-compatible imports and APIs, currently, LiveKit's OpenAI plugin doesn’t support arbitrary OpenAI-compatible endpoints unless they’re specifically set up. The plugin works with predefined methods like

with_groq()

or

with_cerebras()

, which have built-in base URLs and authentication. To connect with MoonshotAI directly, you have a couple of friendly options: 1. You could modify the LiveKit plugin by adding a dedicated

with_moonshot()

method, similar to the ones already there. This would mean forking the plugin and including Moonshot’s base URL and API key handling. 2. Alternatively, you can override the

llm_node()

method in your agent class to manually integrate MoonshotAI. This way, you’d bypass the plugin and use Moonshot’s OpenAI-compatible API directly. I’d be happy to walk you through either of these options step-by-step. Just let me know which path sounds best for you, and we’ll take it from there!

with_moonshot() how would that work?

I'd be happy to help you implement a

with_moonshot()

method! To do this, you'll want to update the LiveKit OpenAI plugin to support MoonshotAI as a provider. Here's a friendly step-by-step guide to get you started: 1. Fork the Plugin: Begin by cloning the LiveKit OpenAI plugin repository to your local setup. 2. Add the Method: Next, create a new static method called

with_moonshot()

inside the plugin's LLM class. This method will set Moonshot's base URL and handle fetching the API key. Here's an example you can follow:

class LLM:
    @staticmethod
    def with_moonshot(opts: dict = None) -> LLM:
        opts = opts or {}
        opts['api_key'] = opts.get('api_key') or os.getenv('MOONSHOT_API_KEY')
        if not opts['api_key']:
            raise ValueError("Moonshot API key is required")

        return LLM(
            model=opts.get('model', 'default-model'),
            base_url=os.getenv('MOONSHOT_BASE_URL'),
            api_key=opts['api_key'],
            **opts
        )

1. Test the Integration: Give the new

with_moonshot()

method a try in your voice processing pipeline to make sure everything runs smoothly. 2. Deploy: Once you're happy with it, you can use your updated plugin in your project or even submit a pull request to the official repository so others can benefit too. This process assumes you're comfortable with Python and familiar with the plugin’s codebase, but please don’t hesitate to ask if you’d like me to walk you through any part or provide more examples. I’m here to help!