In my code , for triggering function tools its taking time ?

Thanks for reaching out, Varchasva. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

@function_tool() async def submit_requirement( self, context: RunContext, customerRequirement: str, fullAddress: str, ) -> str: """ MANDATORY TOOL: Call this IMMEDIATELY when you have both: 1. A valid service requirement from the user 2. City and area(or use last saved location if fullAddress is None) OR when human agent instructs to send details on WhatsApp. Do NOT wait for confirmation. Do NOT announce you're calling it. Just call it silently when both pieces of information are collected. Parameters: - customerRequirement: The service the user needs (e.g., "Plumber", "AC Repair", "Tiffin Service") - fullAddress: The city and area (e.g., "Noida Sector 62", "Mumbai Andheri"). If None, uses last saved location. This tool sends the vendor details to user WhatsApp automatically. """ # If human agent is present, DO NOT generate reply (we're muted anyway) human_agent_present = self.shared_state.get("human_agent_present", False) if not human_agent_present: # Only speak if human agent is not present await context.session.generate_reply( instructions="Instruct the user that their requirement is being processed and they will receive a WhatsApp message shortly." ) # Use last saved location if fullAddress is not provided if not fullAddress or fullAddress.lower() in ["no", "none", "null"]: fullAddress = self.shared_state.get("last_fullAddress") or self.last_city_area if not fullAddress: fullAddress = "No" await self._send_inbound_response_to_round_robin( self.customer_phone_number, customerRequirement, fullAddress, ) # Only end call if human agent is not present if not human_agent_present: await self.end_call(context) else: print("✅ Requirement submitted via WhatsApp (human agent present - call continues)") return "Details have been sent on whatsapp" This is my sample function tool

Thanks — this is helpful. I can walk through likely causes and targeted checks for the sample

submit_requirement

tool you shared, and then recommend concrete steps to measure and reduce the delay. A few quick observations about your code that may impact perceived latency • The tool is async, which is good. But delays can come from before the function is invoked (LLM decision/VAD/TTFT) or from inside the function (synchronous/blocking or slow network calls). • You call

await context.session.generate_reply(...)

which triggers TTS/response generation — if that calls the LLM or TTS provider synchronously it can add delay. •

_send_inbound_response_to_round_robin

and

end_call

likely perform I/O (network/DB/WhatsApp API). If those are synchronous or have long round-trip times, the tool will appear slow. • The code uses shared state lookups and simple logic which are negligible, so focus on external services and LLM/VAD timing. To diagnose the root cause, please provide the following details: 1. Are you seeing delay before entering

submit_requirement

(i.e., time from user input end → tool invocation) or inside

submit_requirement

after it starts? 2. If possible, share timestamps or logs around these points: when user input ends, when the LLM returns a tool decision, when

submit_requirement

starts, and when it completes. Even a short example with elapsed ms is useful. 3. Which LLM provider/model and STT/TTS providers are you using? (Provider + model name.) 4. Is input voice-based with VAD/endpointing enabled? If so, what endpointing/VAD settings are configured? 5. Where is your agent deployed (cloud region, instance type — e.g., AWS t3/t4 or other)? 6. What does

_send_inbound_response_to_round_robin

do? Does it make external HTTP calls (e.g., WhatsApp API) synchronously? Any known latencies there? 7. Do you have TTFT, request/response latencies, or CPU/network metrics you can share? Immediate checks and quick mitigations you can try now • Add lightweight timing logs at the top and bottom of

submit_requirement

and around each awaited call to measure where time is spent. Example timestamps to log: ◦ function enter ◦ before and after

context.session.generate_reply

◦ before and after

_send_inbound_response_to_round_robin

◦ before and after

end_call

• If

_send_inbound_response_to_round_robin

calls external services, make those calls async with short timeouts and retries, or offload them to a background worker/queue so the tool returns quickly. • If the delay is in the LLM decision (TTFT): ◦ Try a lower-latency model or run inference closer to your region. ◦ Reduce VAD endpoint hold times if voice input is used (shorter silence threshold). ◦ Measure Time To First Token to confirm model/provider latency. • If the delay is in TTS or

generate_reply

, try returning the tool result immediately and generating user-facing messages asynchronously (or stream partial replies if supported). If you share the timing logs or answer the diagnostic questions above, I’ll: • Pinpoint whether the bottleneck is LLM/VAD, network, or tool execution. • Suggest exact code changes (example snippets) to make the tool non-blocking or reduce wait time. • Recommend deployment or provider changes if the latency is infrastructure-related. Which part would you like me to help with first — adding timing logs to the function, or reviewing

_send_inbound_response_to_round_robin

for blocking calls?

session = AgentSession( turn_detection=MultilingualModel(), llm=google.realtime.RealtimeModel( voice="Aoede", temperature=0.2, thinking_config=types.ThinkingConfig( include_thoughts=False, ), ), preemptive_generation=True, ) This is the agent session I am using

Thanks — this is helpful. You’re using a Google Realtime LLM with

preemptive_generation=True

and a low temperature, which narrows where delays may come from. I’ll help you pinpoint the bottleneck and give targeted next steps. First, quick summary of what I’ll help with: • Determine whether the delay happens before the tool is invoked (VAD → LLM decision / TTFT) or inside

submit_requirement

(network I/O, blocking calls). • Provide concrete, minimal logging you can add to measure where time is spent. • Suggest immediate changes you can try based on the most likely causes for your setup. Please run the following checks and share the results (small logs or elapsed times are enough): 1) Add timing logs to locate the bottleneck • Log timestamps (or elapsed ms) at these points: ◦ When user audio ends / when you detect end-of-utterance ◦ When the LLM request is sent ◦ When the LLM returns a tool decision (when

submit_requirement

starts) ◦ At the top of

submit_requirement

(function enter) ◦ Before and after

context.session.generate_reply

◦ Before and after

_send_inbound_response_to_round_robin

◦ Before and after

end_call

• You can print ISO timestamps or log time differences (e.g., ms). Share one trace of a slow invocation. 2) Quick checks you can run right now • Is input voice-based with VAD? If yes, reduce endpointing silence threshold temporarily (shorter hold time) and test if latency improves. • Measure Time To First Token (TTFT) for Google Realtime for a simple prompt (no tool call). If TTFT is >300–500ms, provider latency is a likely culprit. • If

_send_inbound_response_to_round_robin

calls WhatsApp or other external APIs synchronously, try swapping it to a quick enqueue to a background worker (example: push job to queue and return immediately) and see if perceived delay disappears. 3) Likely suspects given your session config • LLM TTFT or network latency between your agent and Google Realtime. Preemptive generation reduces some wait but TTFT can still dominate. • VAD/endpointing settings delaying when the agent starts an LLM request. • Synchronous external calls inside

_send_inbound_response_to_round_robin

or

end_call

(WhatsApp API/DB/network). • Rare: CPU throttling on burstable instances if deployed on t3/t4 — check CPU credit exhaustion if applicable. 4) If you want, try a quick experiment • Temporarily modify

submit_requirement

to skip

context.session.generate_reply

,

_send_inbound_response_to_round_robin

, and

end_call

, and just log entry/exit. If tool invocation becomes instant, the delay is inside those calls or LLM decision path; if not, it’s before invocation (VAD/LLM). 5) If you share one sample trace (the timestamps from step 1) I will: • Identify which segment (VAD, LLM request/TTFT, or tool execution) is the main delay. • Give exact code snippets to make the tool non-blocking (example: enqueue WhatsApp send, make

generate_reply

async or deferred, set timeouts). • Recommend provider or deployment changes if it’s TTFT/network related. Which check can you run first — the timing logs or the quick experiment to stub out external calls? I’ll provide the exact minimal code/log lines to add.