To implement your custom LLM with API integration in the LiveKit Agents framework, you can follow these steps:
1. If your API follows the OpenAI chat completions API format, you can use the OpenAI API compatibility layer. This is a straightforward way to integrate many LLM providers.
2. For a completely custom implementation, you can directly implement the LLM node. This involves creating a method that configures the LLM instance with your API's specific requirements, such as:
◦ Accepting model configuration parameters
◦ Handling API key retrieval (from arguments or environment variables)
◦ Setting the appropriate base URL for the API
◦ Returning an LLM instance configured for your API
If you need further guidance or examples, feel free to ask!
Sources:
Large language model (LLM) integrations | LiveKit Docs |
livekit.plugins.openai API documentation