I am having an issue of Open AI LLM gpt 4o mini la...
# ask-ai
a
I am having an issue of Open AI LLM gpt 4o mini latency of 1s minimum. I am calling the API from Pakistan Region and i think request is going ti US which is casuing latency. How can i fix this
t
Thanks for reaching out, Ali. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m responding to your latency concern calling OpenAI’s GPT‑4o‑mini from Pakistan. Short answer: GPT‑4o‑mini calls routed to the US will add ~1s+ latency. To reduce it, use LiveKit Inference (it routes via LiveKit’s proximate data centers and private backbone) or deploy your agent nearer to Pakistan (e.g., India / Middle East) so requests don’t travel to the US. How to proceed (pick one): • Use LiveKit Inference: a. Switch your LLM calls to LiveKit’s Agents/Inference interface (example:
AgentSession(llm="openai/gpt-4o-mini")
) so LiveKit routes requests through its regional infrastructure. b. Verify the agent is provisioned in a region close to Pakistan. c. Measure Time To First Token (TTFT) after switching. • If you must call OpenAI directly: ◦ Run a lightweight proxy/edge service in a nearby cloud region (India/Middle East) to reduce round trips. ◦ Monitor TTFT and network RTT from your location to the OpenAI endpoint to confirm improvements. What I need from you to give exact steps: • Are you already using LiveKit Inference/Agents or calling OpenAI directly? • Which cloud region (or provider) you can deploy in (e.g., AWS Mumbai, GCP Mumbai, Azure UAE)? Tell me those two details and I’ll give exact config steps and sample code for your environment. Sources: Introducing LiveKit Inference: A unified model interface for voice AI | Models | LiveKit Docs | Agents Framework | livekit/livekit_composite | DeepWiki | examples/voice_agents | livekit/livekit_composite | DeepWiki