### Answer To integrate your own LLM with Retell AI, you'll set up a websocket server that connects to Retell's API. [(1) ]() Best Practice: Your LLM's time to first sentence latency (time to first token + time to generate a sentence) is crucial for overall experience smoothness, so use a low-latency LLM inference. Setup Process: The easiest way is to use the provided demo repositories and adapt them to your needs: For Node.js (with Express.js) : The demo includes examples for Azure OpenAI, OpenAI, and OpenRouter. Here's a basic structure for connecting to your LLM client:

typescript
import {
  OpenAIClient,
  AzureKeyCredential,
  ChatRequestMessage,
  GetChatCompletionsOptions,
} from "@azure/openai";
import { WebSocket } from "ws";

You'll need to create a class that handles the LLM connection:

typescript
export class DemoLlmClient {
  private client: OpenAIClient;
  constructor() {
    this.client = new OpenAIClient(
      process.env.AZURE_OPENAI_ENDPOINT,
      new AzureKeyCredential(process.env.AZURE_OPENAI_KEY) ,
    ) ;
  }

For Python (with FastAPI) : Python examples with OpenAI are available in the demo repository. You can customize the prompt, use different LLM providers, add RAG, implement internal states, or create dynamic prompts as needed. Check out the demo repos for more examples: - Node.js: [Azure OpenAI]() , [OpenAI]() , [OpenRouter]() - Python: [OpenAI]()

​ To learn more, see the sources I considered: [[1] Integrate LLM](https://docs.retellai.com/integrate-llm/integrate-llm) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.