I see that we can pass a lot of parameters for openai gpt-oss model in groq. But I do not see any those parameters in groq plugin. WHy? and How can I pass those parameters for the model

Thanks for reaching out, Harsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I want to pass parameters like reasoning, stream,

Thanks — I’m addressing how you can pass the Groq-specific

reasoning

and

stream

parameters when using the LiveKit Groq plugin. Short answer • The plugin exposes common OpenAI-style fields but you can send Groq-only parameters by placing them in

extra_kwargs

(Python) or

modelOptions

(Node.js). Those fields are forwarded to Groq’s API, so you can include

reasoning

,

stream

, or similar parameters there. Examples 1) Python (using

extra_kwargs

)

from livekit.plugins import groq

session = AgentSession(
    llm=groq.LLM(
        model="gpt-oss-120b",
        temperature=0.2,
        extra_kwargs={
            "reasoning": {"type": "chain_of_thought"},  # example shape — confirm with Groq docs
            "stream": True
        }
    ),
)

2) Node.js (using

modelOptions

with OpenAI plugin + Groq)

import * as openai from '@livekit/agents-plugin-openai';

const session = new voice.AgentSession({
    llm: openai.LLM.withGroq({
        model: "gpt-oss-120b",
        temperature: 0.2,
        modelOptions: {
            reasoning: { type: "chain_of_thought" }, // example shape — confirm with Groq docs
            stream: true
        }
    })
});

Important notes • Confirm the exact parameter names and expected value shapes in Groq’s model docs (for example whether

reasoning

expects a string, enum, or object like

{type: ...}

). •

stream: true

typically enables streaming on the provider side. Ensure your LiveKit usage path supports streaming from the LLM; if LiveKit Agents handles responses synchronously in your flow, you may need to adapt how you consume streamed tokens. • The plugin does not validate provider-specific keys. If Groq rejects a parameter, you’ll get an error back from their API — test one parameter at a time if you want to iterate safely. If you want, tell me: • The exact

reasoning

setting value you plan to use (example:

"chain_of_thought"

,

"yes"

, or a config object), and • Whether you need token-level streaming consumed inside LiveKit (so I can confirm any additional changes you might need in your session handling). I’ll produce a tailored code snippet and small test plan based on that.

These are the parameters present in curl-

"stream": true,
         "reasoning_effort": "medium",
         "stop": null

Thanks — I understand you want to pass the exact Groq parameters you use in curl (

stream

,

reasoning_effort

, and

stop

) through the LiveKit Groq plugin. You can do that by forwarding them as raw provider options using

extra_kwargs

(Python) or

modelOptions

(Node.js). Those fields are forwarded to Groq’s API unchanged. Below are concrete examples and important notes. Python example

from livekit.plugins import groq

session = AgentSession(
    llm=groq.LLM(
        model="gpt-oss-120b",
        temperature=0.2,
        extra_kwargs={
            "stream": True,
            "reasoning_effort": "medium",
            "stop": None
        }
    ),
)

Node.js example (OpenAI plugin with Groq)

import * as openai from '@livekit/agents-plugin-openai';

const session = new voice.AgentSession({
    llm: openai.LLM.withGroq({
        model: "gpt-oss-120b",
        temperature: 0.2,
        modelOptions: {
            stream: true,
            reasoning_effort: "medium",
            stop: null
        }
    })
});

Important notes • Parameter shapes: Use the exact names/values Groq expects. In your curl example

reasoning_effort

is a string like

"medium"

,

stream

is a boolean, and

stop

can be

null

or an array/string of stop sequences. Confirm those types in Groq’s docs if you need a different shape. • Streaming behavior: Setting

"stream": true

enables provider-side streaming. Ensure your LiveKit Agent flow (and any client code consuming responses) can handle streamed tokens. LiveKit may present responses synchronously in some flows, so test how streamed tokens are delivered to your session handler. • Validation: The plugin does not validate provider-specific keys. If Groq rejects a parameter, you’ll receive an error response from Groq’s API. If you want to test safely, send one parameter at a time. •

stop

values: If you intend to stop on specific tokens/strings, pass an array like

["\n\n", "STOP"]

instead of

null

. Use

null

only if you want no stop sequences. • Error debugging: If a parameter fails, capture the Groq API error payload (status and message). Share it here and I’ll help interpret it. If you’d like, tell me: • Whether you need token-by-token streaming handled inside LiveKit (so I can give code for consuming streamed tokens), and • The exact

stop

sequences you plan to use (or confirm you want no stops). I’ll then provide a tailored snippet and a short test plan.

how are these parameters passed as model config then?

Thanks — I’ll explain exactly how those Groq parameters become part of the model configuration when using the LiveKit Groq plugin, and show a concrete example. Acknowledgement • You asked how parameters like

stream

,

reasoning_effort

, and

stop

are passed as model config when using the LiveKit Groq plugin. How it works (short) • When you initialize the

LLM

in LiveKit Agents, any provider-specific options you put in

extra_kwargs

(Python) or

modelOptions

(Node.js) are merged into the request body sent to Groq’s Chat Completions API. • The plugin sends the standard OpenAI-style fields plus the contents of

extra_kwargs

/

modelOptions

unchanged to Groq. This is the mechanism that lets you pass

stream

,

reasoning_effort

,

stop

, etc., even though they aren’t top-level LiveKit fields. What to pass and where • Python: add provider-specific keys in

extra_kwargs

on the

groq.LLM

constructor. • Node.js: add provider-specific keys in

modelOptions

(for example via

openai.LLM.withGroq()

). Concrete examples 1) Python

from livekit.plugins import groq

session = AgentSession(
    llm=groq.LLM(
        model="gpt-oss-120b",
        temperature=0.2,
        extra_kwargs={
            "stream": True,
            "reasoning_effort": "medium",
            "stop": None
        }
    ),
)

2) Node.js (OpenAI plugin with Groq)

import * as openai from '@livekit/agents-plugin-openai';

const session = new voice.AgentSession({
    llm: openai.LLM.withGroq({
        model: "gpt-oss-120b",
        temperature: 0.2,
        modelOptions: {
            stream: true,
            reasoning_effort: "medium",
            stop: null
        }
    })
});

Implementation notes and tips • The plugin does not validate or transform these keys — they are forwarded directly to Groq. Use the exact names and value shapes Groq expects. • For

stop

, you can pass

null

, a string, or an array of strings (e.g.,

["\n\n", "STOP"]

) depending on the Groq API expectations. • For

stream: true

, Groq will stream tokens. Ensure your client flow (how LiveKit Agents consumes LLM responses) supports streaming; otherwise you may receive the full response only after the stream completes. • If Groq rejects a parameter, you’ll receive an error from the Groq API. If that happens, capture the error payload (status & message) and I can help interpret it. • If you want to test incrementally, add one provider-specific key at a time so errors are easier to diagnose. If you’d like, tell me: • whether you need token-level streaming inside your LiveKit flow (I can show how to consume streamed tokens), and • an example

stop

value you intend to use (so I can show the exact shape to pass). Sources: Groq LLM plugin guide | LiveKit Documentation | OpenAI LLM models | LiveKit Documentation

This is how I am initializing LLM - \

groq.LLM(
  model='openai/gpt-oss-120b',
    temperature=0.6,
    api_key=config['GROQ_API_KEY'],
    extra_kwargs={
        "stream":True,
        "reasoning_effort":"medium",
        "stop": None
    }
)

And I am getting an error - Error in entrypoint function: LLM.__init__() got an unexpected keyword argument 'extra_kwargs'

Thanks — I see the error you’re hitting and why it’s happening. I’ll explain what’s going on and give the correct options and code examples so you can pass

stream

,

reasoning_effort

, and

stop

without the constructor error. What’s happening • The

groq.LLM

plugin’s constructor does not accept

extra_kwargs

. That parameter is not supported on

groq.LLM

, so Python raises:

LLM.__init__() got an unexpected keyword argument 'extra_kwargs'

. • The Groq plugin only accepts its documented top-level args (for example:

model

,

temperature

,

tool_choice

,

parallel_tool_calls

).

api_key

should come from the

GROQ_API_KEY

environment variable, not passed into the constructor. How to pass provider-specific options You have two valid approaches depending on the model/provider you intend to use: 1) Use groq.LLM only (Groq-hosted models) • If you are calling a Groq-hosted model (e.g.,

llama3-8b-8192

), use

groq.LLM

but you cannot pass

extra_kwargs

in the constructor. Only supply the plugin’s supported parameters. • Example (no provider passthrough):

from livekit.plugins import groq
import os

os.environ["GROQ_API_KEY"] = "your-groq-key"

llm = groq.LLM(
    model="llama3-8b-8192",
    temperature=0.6
)

• This will not let you directly set

reasoning_effort

,

stream

, or

stop

via

groq.LLM

constructor. 2) Use inference.LLM (passthrough / cross-provider + extra_kwargs) • If you need to pass Groq-specific keys like

stream

,

reasoning_effort

, or

stop

, use LiveKit’s

inference.LLM

(the inference gateway) which supports

extra_kwargs

. This routes parameters through the inference layer to the underlying provider. • Use the provider model ID you intended (e.g.,

openai/gpt-oss-120b

if that’s your target) or the Groq model ID if routing to Groq via inference. • Example:

from livekit.agents import inference

llm = inference.LLM(
    model="openai/gpt-oss-120b",  # or a Groq model id if you want Groq via inference
    extra_kwargs={
        "stream": True,
        "reasoning_effort": "medium",
        "stop": None
    },
    temperature=0.6
)

• This will forward

extra_kwargs

to the provider API (Groq or other) as-is. Which to choose • If you must use Groq-specific parameters and need the exact control you have in your curl calls, use

inference.LLM

with

extra_kwargs

. • If you are happy to use only the parameters the Groq plugin exposes directly and are calling a Groq-hosted model, stick with

groq.LLM

and the plugin’s supported args. Quick checklist to avoid errors • Confirm the model ID belongs to the provider you intend to call (Groq vs openai/gpt-oss). • If using

groq.LLM

, do not include

extra_kwargs

in its constructor. • If you need passthrough parameters, switch to

inference.LLM

and put the parameters into

extra_kwargs

. • Ensure API keys are set in environment variables expected by the plugin (e.g.,

GROQ_API_KEY

). If you want, tell me: • Which exact model ID you want to call now (e.g.,

openai/gpt-oss-120b

or a Groq model like

llama3-8b-8192

), and • Whether you need real-time token streaming consumed by your client. I’ll give the precise snippet and a short test plan for your chosen path. Sources: Groq LLM plugin guide | LiveKit Documentation | Groq and LiveKit | LiveKit Documentation | Baseten LLM plugin guide | LiveKit Documentation | OVHCloud LLM plugin guide | LiveKit Documentation | Google Gemini LLM | LiveKit Documentation