Gemini 3 Flash function calling issue Using goo...
# ask-ai
a
Gemini 3 Flash function calling issue Using google.LLM(model="gemini-3-flash-preview") with livekit-agents==1.3.9 and livekit-plugins-google==1.3.9. Problem: First function call succeeds, but immediately throws: 400 Bad Request: "Please ensure that function call turn comes immediately after a user turn or after a function response turn." Then every subsequent turn errors with the same message. Agent gets stuck. Context: The function calls ARE coming after user turns (initial greeting, Continue button clicks), but Gemini rejects the conversation structure anyway. Same code works fine with OpenAI. Is this a known Gemini 3 + function calling bug in 1.3.9? Possibly related to how the plugin formats chat context or handles thought signatures?
t
Thanks for reaching out, Jon. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’m sorry you’re blocked by that error. I’m responding about the Gemini 3 Flash function-calling issue you described (first call succeeds, then immediate 400: "Please ensure that function call turn comes immediately after a user turn or after a function response turn", repeated thereafter). I don’t have a confirmed known-bug entry for Gemini 3 Flash + livekit-agents 1.3.9 / livekit-plugins-google 1.3.9. To help narrow this down quickly, please provide the following so I can reproduce and pinpoint the cause: 1. A minimal reproducible conversation log (full sequence of chat messages and role labels — user/assistant/function — from session start through the failing turn). Redact any sensitive data. 2. The exact request payload sent to Google for the first successful function call and for the failing subsequent call(s) (headers + body or the equivalent client call). 3. The code snippet showing how you format and send chat context to the Google plugin/LLM (especially how you append function responses, user turns, and any “thought” or signature metadata). 4. Which function-calling mode you’re using (explicit function call vs. model-invoked function call) and whether you pass an explicit
function_call
parameter. 5. Whether you see any differences in message ordering or role names between the OpenAI case (works) and Gemini case (fails). Quick troubleshooting steps you can try now (these often reveal what’s malformed): • Verify role/order strictly: ensure the immediate previous message before the function call is a user message or a function response message (no assistant non-function messages, no system messages between them). Gemini enforces this ordering. • Strip any extra metadata from assistant messages (e.g., thought signatures, custom fields). Try sending only {role, content} entries to see if the model accepts the function call. • Compare the exact JSON sequence you send to OpenAI vs Google for the same interaction. Look for differences in role naming (
assistant
vs
tool
), extra assistant messages, or duplicate assistant turns. • If you include a function response message, ensure it is labeled with role
function
and includes content exactly where the next call is expected. • Temporarily disable any plugin-specific wrappers that mutate the chat history (one at a time) to isolate whether the plugin formatting causes the issue. What I’ll do once you provide logs/snippets: • Reproduce locally against Gemini 3 Flash with the same request format you’re using. • Check whether the plugin adds assistant-thought metadata or transforms roles that could break Gemini’s strict turn-order rule. • Suggest a minimal patch or workaround (strip/translate roles, reorder messages, or adjust function_call parameter) you can test immediately. If you can paste the minimal log + request payload next, I’ll start reproducing and give a concrete fix.
a
Found it! System messages are breaking Gemini's turn order. Sequence that fails: 1. Function call: present_selection_cards() - works 2. System message inserted: "Selection Cards is now visible to the user" 3. Function output returned 4. Next LLM call → Error: "function call turn comes immediately after user turn..." The system messages (tool state notifications) are being added to chat context between function calls and user turns, which Gemini rejects but OpenAI allows. Is there a way to exclude system messages from the chat context sent to Gemini, or mark them differently so Gemini doesn't see them as breaking turn order?
t
Great find — you nailed the cause. You're asking how to stop system messages from breaking Gemini's strict turn order so function calls work consistently. Options you can use now: • Filter out system messages before sending context to Gemini. a. Build a filtered chat context that excludes messages where
role == "system"
. b. Use that filtered context when calling Gemini so the assistant/function/user turn ordering remains intact. • Store those notifications outside the LLM chat history. ◦ Put tool/state notifications in session or agent metadata, or publish them via LiveKit data messages (participant.publish_data()), instead of inserting them as
system
messages in the chat stream. • If you need the agent to be aware of the state, pass a short sanitized summary as a user or assistant message only at safe points (e.g., after a user turn), not between a function call and the next function call. If you want, paste a minimal snippet of how you currently append system messages to the chat context and I’ll show the exact place to apply the filter and a one-line patch you can drop in. Sources: Data packets | LiveKit docs | Web and mobile frontends | LiveKit docs | Google Gemini LLM plugin guide | LiveKit docs
a
i wonder if that filtering also needs to happen for gemini 2.5 live
t
Good question — I don’t have confirmation that Gemini 2.5 Live enforces the same strict turn-order checking as Gemini 3. Because that information isn’t available, the safest approach is to treat Gemini 2.5 Live as potentially sensitive to intermediary system messages and apply the same filtering. Recommendations: • Apply the same filter that removes
role == "system"
messages before sending context to Gemini 2.5 Live. • If you need those notifications, keep them in session/agent metadata or publish via LiveKit data messages instead of inserting them into the chat history. • Test quickly by running one session with system messages included and one with them filtered to see if behavior differs. If you want, share a minimal example of your 2.5 request payload and I’ll point out exactly where to filter. Sources: Gemini Live API plugin | LiveKit docs