hello

Thanks for reaching out, Joshua. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

so im using the google realtime model and its the latest one. I want to know why this error occurs post tool call after the tool call executes the data is added to the context but then after the success this error follows 2025-12-16 135832,947 - DEBUG livekit.agents - tools execution completed {"speech_id": "speech_a15cf03d0282"} 2025-12-16 135833,716 - ERROR livekit.plugins.google - error in receive task: received 1007 (invalid frame payload data) Invalid JSON payload received. Unknown name "scheduling" at 'tool_response.function_responses[0]': Cannot find field.; then sent 1007 (invalid frame payload data) Invalid JSON payload received. Unknown name "scheduling" at 'tool_response.function_responses[0]': Cannot find field. Traceback (most recent call last): File "C:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\.venv\Lib\site-packages\livekit\plugins\google\realtime\realtime_api.py", line 761, in _recv_task async for response in session.receive(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<31 lines>... self._num_retries = 0 # reset the retry counter ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\.venv\Lib\site-packages\google\genai\live.py", line 453, in receive while result := await self._receive(): ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\.venv\Lib\site-packages\google\genai\live.py", line 534, in _receive raw_response = await self._ws.recv(decode=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\.venv\Lib\site-packages\websockets\asyncio\connection.py", line 322, in recv raise self.protocol.close_exc from self.recv_exc websockets.exceptions.ConnectionClosedError: received 1007 (invalid frame payload data) Invalid JSON payload received. Unknown name "scheduling" at 'tool_response.function_responses[0]': Cannot find field.; then sent 1007 (invalid frame payload data) Invalid JSON payload received. Unknown name "scheduling" at 'tool_response.function_responses[0]': Cannot find field. 2025-12-16 135833,725 - DEBUG livekit.plugins.google - send task finished. 2025-12-16 135833,730 - DEBUG livekit.plugins.google - connecting to Gemini Realtime API... [Audio] Array (Intel® Smart [-120.00 dBFS] [------------------------------]

Thanks — you're asking why the Gemini Realtime session closes with "Unknown name 'scheduling'" after a tool call even though the tool executed successfully. You're correct to focus on the tool response payload — the Realtime API is rejecting the JSON it receives. What’s happening • The Google Realtime (Gemini Live) API expects tool responses to follow a specific schema for function/tool outputs. • Your tool response includes a field named

scheduling

inside

tool_response.function_responses[0]

, which Gemini does not recognize. That causes the websocket to close with the 1007 invalid frame payload error. How to fix it 1. Inspect the tool return value right after the tool completes (before it’s sent to Gemini) and confirm its top-level fields. 2. Remove or rename

scheduling

(and any other custom fields) from the object you send back. Return either: ◦ a plain string, or ◦ a dictionary/object that only contains the valid keys expected by the FunctionCallOutput schema. 1. If you need to keep structured metadata (like scheduling info), move it into a separate internal context or log rather than embedding it into the tool response sent to Gemini. 2. Search your agent/tool code path for where the tool result is packaged for the Realtime session — ensure no middleware or serializer injects extra fields into

tool_response.function_responses

. 3. Re-run and confirm the websocket no longer receives unknown fields; the 1007 error should disappear. If you want, share: • the exact tool return object (redact sensitive data) or the function that packages

tool_response.function_responses[0]

, and • where

scheduling

is added (tool code or post-processing), and I’ll point exactly where to remove or relocate that field. Sources: Tool definition and use | LiveKit docs | Gemini Live API plugin | LiveKit docs

if response.status_code == 200 and response.json():                     patient = response.json()[0]                     userdata.patient_status = "existing_patient"                     patient_id = patient.get("id") or patient.get("patient_id")                     userdata.external_patient_id = patient_id                                         # Extract email and fax from patient record                     patient_email = patient.get("email") or patient.get("Email")                     patient_fax = patient.get("fax") or patient.get("Fax")                                         if patient_email:                         userdata.email = patient_email                     if patient_fax:                         userdata.fax_number = patient_fax                                         logger.info(f"✅ Patient found - ID: {patient_id}, Email: {patient_email}, Fax: {patient_fax}")                 else:                     userdata.patient_status = "not_found"                     logger.warning(                         f"⚠️ Patient not found - Status {response.status_code}: {response.text}"                     )

Thanks — you shared the patient-handling snippet. This looks like the code that builds the tool's return data after an API lookup. To stop the Gemini Realtime 1007 error you must ensure the object you ultimately send as the tool/function response does not include a

scheduling

field. Quick checklist and targeted suggestions based on your snippet: 1. Confirm where the final tool response is constructed • After you set

userdata

fields, find the code that serializes or returns the tool result to the agent/framework. That object is what Gemini inspects. 1. Remove/relocate

scheduling

• If

userdata

or the returned object contains a

scheduling

attribute, remove it before returning, or move it to a separate metadata/log payload that is NOT sent in

tool_response.function_responses

. • Example: build a clean result dict like:

result = {
  "patient_status": userdata.patient_status,
  "external_patient_id": userdata.external_patient_id,
  "email": userdata.email,
  "fax_number": userdata.fax_number
}
# return/send `result` to the Realtime API

1. Defensive sanitize step (recommended) • Right before sending to Gemini, run a sanitizer that drops unexpected keys:

allowed = {"patient_status","external_patient_id","email","fax_number"}
clean = {k:v for k,v in result.items() if k in allowed}

1. If

scheduling

is added elsewhere • Search for any middleware/serializer or post-processing that might inject

scheduling

(hooks, loggers, or agent tool wrappers). Remove it from the payload there. If you want, paste the code that packages/returns the tool response (the function that actually returns data to the agent or calls the Realtime send), and I’ll mark exactly where to strip

scheduling

.

# Build response message for LLM (match prescription agent pattern) message_parts = [f"Patient found: {first_name_str} {last_name_str}, DOB: {normalized_dob}."] if patient_email: message_parts.append(f"Email on file: {patient_email}.") else: message_parts.append("No email on file.") if patient_fax: message_parts.append(f"Fax on file: {patient_fax}.") else: message_parts.append("No fax on file.") # Return dict with success/message keys (prescription agent pattern) return { "success": True, "message": " ".join(message_parts) } finally: if detector: detector.set_tool_active(False)

You're close — the message-building return looks fine. The remaining problem is that some object sent to Gemini still contains a

scheduling

field. I can help you find and remove it. Quick next steps — please confirm or share one of these so I can point to the exact spot to change: • The function that packages/returns the tool response to the agent / Realtime session (the code that actually does the

return

or calls the framework send). • If you don't have a single packaging function, tell me whether you return

userdata

,

result

, or the dict you showed directly to the agent. • A short dump (redact sensitive data) of the final object right before it's sent to Gemini. If you prefer a direct change now, add this sanitizer immediately before returning/sending the response:

# allowed keys for Gemini function/tool output
allowed = {"success","message","patient_status","external_patient_id","email","fax_number"}
final = {k:v for k,v in result.items() if k in allowed}
# return/send `final` instead of `result` or `userdata`

Send the packaging/return function or the final object and I'll mark exactly where to strip

scheduling

.

Perfect! The logs show the tool is returning the CORRECT format:

🔍 TOOL RESPONSE BEING RETURNED: {'success': True, 'message': 'Patient found: [redacted]...'}
🔍 RESPONSE TYPE: <class 'dict'>
🔍 RESPONSE KEYS: ['success', 'message']

But 0.7 seconds later, Google Gemini rejects it with

"Unknown name 'scheduling'"

. This means the LiveKit framework is adding something between our return and Gemini. Let me check the LiveKit Google plugin's tool response packaging: KEY DIFFERENCE FOUND! Prescription agent has NO return type annotation:

python


async def fetch_medical_record(
    self,
    requester: str,
    first_name: str,
    ...
):  # ← NO return type

Medical records agent originally had

-> str:

which I removed, but let me check if there's still something: Good - no return type annotation and no "scheduling" in the agents. The issue must be in how LiveKit/Google SDK serializes the function declaration. Let me check the function tool imports: Same imports. The issue is that Google Gemini is receiving something with "scheduling" in it but we're not sending it. This suggests the LiveKit SDK is auto-generating function schemas from the tool decorators. Let me check if there are any other tools defined that might have "scheduling" in their schema: Searcheddef.*schedule|def.appointment in c:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\agents (.py) (0) Analyzed c:\Users\rayan\Downloads\Testing-AI\NYGA\nyga-medical-records-v1\agents Wait - the error says

'tool_response.function_responses[0]'

- this is the response format, not the function declaration. Google expects a specific response schema. Let me check if we need to wrap the response differently: Created outline ai.google.dev Found it! Position 33 mentions "Scheduling" - let me read that: Analyzed content 8 chunks FOUND IT! "Scheduling" is a field in the Google Gemini API for controlling response timing (

SILENT

,

WHEN_IDLE

,

INTERRUPT

). The error means LiveKit is trying to send a field called "scheduling" in the tool response, but it's not allowed there. This is a LiveKit plugin bug - it's adding a "scheduling" field to the tool response that Google Gemini doesn't accept in

function_responses[0]

. Let me check if there's a way to control this or if we need to report it. First, let me see if the prescription agent has any special configuration: Let me check the agent session config where the Google Realtime model is configured: The configuration looks fine. The issue is that LiveKit's Google plugin is incorrectly adding a "scheduling" field to the tool response. This is a LiveKit plugin bug. ROOT CAUSE IDENTIFIED: LiveKit Plugin Bug The error

"Unknown name 'scheduling'"

is caused by the LiveKit Google Realtime plugin incorrectly adding a

"scheduling"

field to the tool response payload sent to Gemini. Evidence: 1. ✅ Our tool returns:

{"success": True, "message": "..."}

2. ✅ Logs confirm:

RESPONSE KEYS: ['success', 'message']

3. ❌ Google rejects with:

"Unknown name 'scheduling' at 'tool_response.function_responses[0]'"

4. 📚 Google docs show "Scheduling" is a separate field for response timing, not allowed in tool responses Workaround: Return Plain String Since the prescription agent works and uses dicts, but we're still getting errors, let's try returning a plain string (the LiveKit plugin might be wrapping dicts incorrectly): dispatch_agent.py +2 -6 dispatch_agent.py +2 -5 support_agent.py +2 -5 ✅ WORKAROUND APPLIED FOR LIVEKIT PLUGIN BUG The Issue: LiveKit's Google Realtime plugin is incorrectly adding a

"scheduling"

field to tool responses, which Google Gemini API rejects with:

Invalid JSON payload received. Unknown name "scheduling" at 'tool_response.function_responses[0]'

Root Cause: • "Scheduling" is a valid Gemini API field for controlling response timing (

SILENT

,

WHEN_IDLE

,

INTERRUPT

) • But it's NOT allowed in

function_responses

- only in other contexts • LiveKit plugin is incorrectly adding it to tool response payloads Workaround Applied: Changed ALL tools to return plain strings instead of dicts:

python


# ❌ BEFORE (caused "scheduling" error):
return {"success": True, "message": "Patient found..."}

# ✅ NOW (workaround):
return "Patient found: [redacted], DOB: [redacted]. Email on file: <mailto:dummy@gmail.com|dummy@gmail.com>."

Tools Fixed: 1. ✅ 2. get_patient_demographics() 3. → Returns plain string 4. ✅ 5. create_records_dispatch_task() 6. → Returns plain string 7. ✅ 8. save_task() 9. → Returns plain string Test the agent now - the "scheduling" error should be gone! 🚀 If this works, the prescription agent must also be returning strings (not dicts) despite what the code shows, OR they're using a different LiveKit/Google plugin version. Feedback submitted

Thanks — this is a great investigation and your conclusion is correct: the LiveKit Google plugin is adding a

scheduling

field into the tool response payload that Gemini rejects. I’ll respond to three things: confirm, validate your workaround, and suggest next steps. Confirmation • Your evidence and root cause are correct: the tool returns a clean dict, but the plugin injects

scheduling

into

tool_response.function_responses[0]

, triggering Gemini’s 1007 invalid payload error. Workaround validation • Changing tools to return plain strings is a valid short-term workaround because it avoids the plugin path that injects the extra field. If your agents only need human-readable responses, keep the string return. • The sanitizer approach you were offered earlier (whitelist keys and strip others just before sending) is still safe if you can intercept the final payload at the plugin boundary. Recommended next steps 1. Keep the string-return workaround in your deployed agents if that unblocks you immediately. 2. Add a sanitizer as a safer short-term fix—place it in the code path that hands the tool result to LiveKit (or wrap the plugin send call) to remove

scheduling

and any other unexpected keys. 3. If you want the richer structured returns preserved long-term, I can help you craft a minimal patch or exact location in the plugin code to remove the injected

scheduling

field before sending to Gemini. Share the LiveKit Google plugin file that serializes

tool_response

(or the small snippet around where

function_responses

is built) and I’ll mark the exact lines to change. 4. If you’d like, I can also suggest a safe structured-to-string adapter that preserves your structured data in logs/internal context while returning strings to Gemini. Would you like me to: • show the sanitizer snippet to place at the plugin boundary, • or review the plugin serialization code and mark the exact edit to remove

scheduling

?

sanitization code please

Below is a concise sanitizer you can run just before the tool response is handed to the LiveKit/Google plugin (or right where

function_responses

is built). It: • removes

scheduling

and any other disallowed keys • preserves allowed structured fields you want to keep (adjust

allowed

as needed) • works for dict responses or lists of responses Place this immediately before the call that sends the result to LiveKit/Gemini.

def sanitize_tool_response(tool_response):
    """
    Ensure the object sent to Gemini contains only allowed keys.
    Accepts dict, list (of dicts), or string.
    Returns sanitized object (string or dict/list).
    """
    # Allowed top-level keys for function/tool output
    allowed = {"success", "message", "patient_status", "external_patient_id", "email", "fax_number"}

    # If already a plain string, return as-is
    if isinstance(tool_response, str):
        return tool_response

    # If it's a list of responses, sanitize each element
    if isinstance(tool_response, list):
        sanitized_list = []
        for item in tool_response:
            if isinstance(item, dict):
                sanitized_list.append({k: v for k, v in item.items() if k in allowed})
            else:
                sanitized_list.append(item)
        return sanitized_list

    # If it's a dict, filter keys
    if isinstance(tool_response, dict):
        return {k: v for k, v in tool_response.items() if k in allowed}

    # Fallback: convert to string to avoid unexpected schema
    return str(tool_response)

Usage example (wrap the send call):

result = fetch_medical_record(...)  # could be dict, list, or str
clean = sanitize_tool_response(result)
# send `clean` to LiveKit/Gemini instead of `result`

If you want, tell me where the send occurs (file/function) and I’ll show the exact two-line change to insert the sanitizer.