Hi team, I'm trying to reduce latency when handing...
# agents
a
Hi team, I'm trying to reduce latency when handing off to an ai agent from an ai agent via @function_tool. I already have the agents prewarmed and stored on a session object, but we are noticing long delays of 4s during the transition. Flow is Human says 'Hello?' , agent1 calls transfer_to_assistant() tool, agent2 calls generate_reply(), and answers. From 'Hello' to the agent2's answer is the latency/delay we are trying to reduce. Any idea what else we can try to make it faster? We are on livekit cloud and hosting our own agents.
l
I suggest taking a look at the observability trace with Agent insights: https://docs.livekit.io/agents/observability/ to see if that offers any clues. Alternatively, you could try and isolate what might be the root cause when compared with our simple multi-agent sample: https://github.com/livekit-examples/python-agents-examples/blob/main/multi-agent/long_or_short_agent.py • Do you see similar latency when running your agent in LK cloud? • Is
generate_reply
a culprit? (It shouldn't be, I just mention this as a difference between that sample provided, and how you say your app works) • Try using a STT / LLM / TTS from LK inference • Or, swapping out for a different STT / LLM / TTS • Something else.