Switched from plugin to LK inference. STT and TTS ...
# inferencem
many-forest-60185
10/04/2025, 9:33 PMSwitched from plugin to LK inference. STT and TTS work fine (Deepgram and Elevenlabs respectively). However, I get the following error with
llm="openai/gpt-4.1-mini"livekit.agents._exceptions.APIStatusError: Error proxying completions: provider: azure model: gpt-4.1-mini-provisioned, message: POST "<https://agent-gateway.cognitiveservices.azure.com/openai/v1/chat/completions>": 400 Bad Request {"message": "Missing required parameter: 'response_format.json_schema'.","type": "invalid_request_error","param": "response_format.json_schema","code": "missing_required_parameter"}many-forest-60185
10/04/2025, 9:43 PMAh, this is likely because I overwrote the LLM node to use structured outputs. I guess the interaction is different for plugins vs inference?
many-forest-60185
10/04/2025, 9:51 PMYeah, seems to be an issue when I try to use structured outputs with LK inference.
d
dry-elephant-14928
10/05/2025, 12:16 AMah thanks for the report. we will take a look.
CC @fancy-lifeguard-70250
f
fancy-lifeguard-70250
10/05/2025, 12:45 AMOn it. I will take a look.
@many-forest-60185 would you be able to send the request JSON? I suspect I know what the problem is but I would to confirm with the JSON. You can redact the messages field if needed. Also, feel free to DM me the JSON if that’s better for you.
m
many-forest-60185
10/05/2025, 1:08 AMSure thing. JSON here.
many-forest-60185
10/05/2025, 1:09 AM{"model": "gpt-4.1-mini","messages": [/* redacted */],"tool_choice": "none","response_format": {"type": "json_schema","json_schema": {"name": "ResponseStructure","strict": true,"schema": {"type": "object","properties": {"message_type": { "type": "string" },"output": { "type": "string" },"input_language": { "type": "string" }},"required": ["message_type", "output", "input_language"],"additionalProperties": false}}}}f
fancy-lifeguard-70250
10/05/2025, 1:16 AMWonderful. I’ll look into this and report back. Thank @many-forest-60185
👍 1
fancy-lifeguard-70250
10/05/2025, 4:54 AMOk, I found the issue, i'll let you know when I have a fix.
🎉 1
fancy-lifeguard-70250
10/06/2025, 5:57 PM@many-forest-60185 I haven't forgotten about you, fix should be going out today. I will let you know.
m
many-forest-60185
10/06/2025, 7:26 PMSounds good
f
fancy-lifeguard-70250
10/06/2025, 9:46 PM@many-forest-60185 Fix is live. Let me know if you experience anything unexpected.
m
many-forest-60185
10/06/2025, 10:13 PM@fancy-lifeguard-70250 Works like a charm. Didn’t even need to update! Thanks for taking care of it so quickly. Eager to see how LK Inference impacts average production latency.. especially on the LLM piece.
f
fancy-lifeguard-70250
10/06/2025, 10:44 PMSick!!! Happy to help and apologies for the inconvenience.
🙌 1