Hi LiveKit team! I'm trying to solve a quite puzzl...
# cloud
f
Hi LiveKit team! I'm trying to solve a quite puzzling situation. I've created a phone agent that should operate in Europe, and I'm wondering why I perceive an excess of latency when deploying my agent on the cloud compared to local hosting. I'm currently using LiveKit Inference since I've seen it should be the best solution to reduce latency. This is the setup:
Copy code
AgentSession[CompleteOrder](    
    userdata=complete_order,
    max_tool_steps=10,
    stt=inference.STT(model="deepgram/nova-3", language="it"),
    llm=inference.LLM(model="google/gemini-2.5-flash"),
    tts=inference.TTS(
        model="cartesia/sonic-3",
        voice="88b329db-85d7-47cc-a5c5-98225a756721",
        language="it",
        extra_kwargs={"speed": 1.2, "volume": 1.2}
    ),
    vad=ctx.proc.userdata["vad"],
    preemptive_generation=True,
)
I don't understand where the problem is coming from. I've tried several troubleshooting steps: • Deployed the provided base agent for comparison • LiveKit's demo agent in multiple configurations: (1) self-hosted locally on my machine, (2) deployed to LiveKit Cloud in Europe, and (3) deployed to LiveKit Cloud in the US. The locally self-hosted demo agent consistently outperformed both cloud deployments, regardless of region. • Tried different versions of LiveKit and Python Can you help me identify where this excess latency might be arising from?🙏
r
I am not familiar with your code so it is hard to say. But can you reproduce the same results using a simple agent with no additional logic. Something like basic_agent.py
But use this configuration in basic agent:
Copy code
stt=inference.STT(model="deepgram/nova-3", language="it"),
    llm=inference.LLM(model="google/gemini-2.5-flash"),
    tts=inference.TTS(
        model="cartesia/sonic-3",
        voice="88b329db-85d7-47cc-a5c5-98225a756721",
        language="it",
        extra_kwargs={"speed": 1.2, "volume": 1.2}
    ),
    vad=ctx.proc.userdata["vad"],
    preemptive_generation=True,
f
I've already done that. I tested the basic_agent.py with that model config. I'm still perceiving higher latency with the cloud deployment. What's particularly strange is that when I deploy the agent in the US, I actually get better performance compared to deploying it in the EU, which seems counterintuitive since I'm currently located in Europe. I would expect the EU deployment to have lower latency given my physical location, but that's not what I'm experiencing. Any ideas what might be causing this?