is there a way to stop model reasoning in for aws models in aws llm plugin config

Thanks for reaching out, arpit. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

yes I want the model to do tool calls but I dont want the models internal reasoning this is for Amazon Nova models

Thanks — you're asking how to let Amazon Nova models make tool calls but suppress their internal reasoning/chain-of-thought. You can’t turn off Nova’s internal reasoning with a single plugin flag, but you can allow tool calls and filter out reasoning before it reaches users (or TTS). Two practical options: • Keep tool calls enabled: ◦ Set

tool_choice="auto"

(or

"required"

) in the AWS LLM config to allow tool usage. • Suppress internal reasoning by filtering LLM output: ◦ Override your agent’s

llm_node

(or wrap the LLM stream) to inspect streamed tokens and drop or replace any chain-of-thought/internal-reasoning fragments (e.g., thinking markers, internal tags, or patterns you observe) before forwarding to TTS or the user. ◦ Example pattern: stream chunks from

self.llm.chat(...)

, skip chunks containing reasoning markers (or strip them), and yield only user-facing content. Optional advanced approaches: • Buffer the model output, run a quick filter/moderation step (a small LLM or rule-based filter) to remove reasoning text, then emit the cleaned output. • If you’re using Nova Sonic (realtime speech-to-speech) mention that and I’ll tailor the stream handling example for realtime models. If you want, share a short sample of the model output (the reasoning tokens you see) or confirm whether you’re using Nova Sonic or a standard Nova LLM, and I’ll provide a specific code snippet to insert into

llm_node

. Sources: Amazon Nova Sonic integration guide | LiveKit docs | Modifying LLM output before TTS | LiveKit docs | LLM-Powered Content Filter | LiveKit docs | Amazon Bedrock LLM plugin guide | LiveKit docs

2025-12-31 083343 | DEBUG | trace_id=6bcc17fad92a485ebcf7ec3b97797ad4 | Conversation item added: type='conversation_item_added' item=ChatMessage(id='item_c3109ce0174b', type='message', role='assistant', content=["<thinking> Elba has provided her name and asked for Arpit's name. Now that Arpit's name has been provided, I should call the

name_provided

tool to acknowledge the name. </thinking>\n"], interrupted=False, transcript_confidence=None, hash=None, created_at=1767170010.8826632) created_at=1767170023.0970216 This is how I am seeing the output

Thanks — I see the debug log showing the assistant emitting "<thinking>…</thinking>" in the message stream. You want tool calls kept but those internal reasoning tags removed. Below are two concise, actionable options and a short code example to filter the "<thinking>" blocks by overriding

llm_node

. Options • Stream filter (preferred): intercept the LLM output stream, drop or strip any fragments containing "<thinking>" before yielding to TTS or the user. • Post-process buffer: collect the model response, remove reasoning blocks with a regex, then emit the cleaned text (useful if you need whole-message context). Example: override

llm_node

to strip "<thinking>...</thinking>" while leaving tool usage enabled

import re

THINK_TAG_RE = re.compile(r"<thinking>.*?</thinking>", flags=re.DOTALL)

async def llm_node(self, chat_ctx, tools, model_settings=None):
    async def process_stream():
        async with self.llm.chat(chat_ctx=chat_ctx, tools=tools, tool_choice="auto") as stream:
            async for chunk in stream:
                if chunk is None:
                    continue
                # extract text content from stream chunk (adjust per chunk shape)
                content = getattr(chunk.delta, "content", None) if hasattr(chunk, "delta") else str(chunk)
                if not content:
                    yield chunk
                    continue
                # remove thinking blocks
                cleaned = THINK_TAG_RE.sub("", content)
                # skip empty results (or yield sanitized chunk)
                if cleaned.strip() == "":
                    continue
                # replace chunk content with cleaned text — adjust depending on chunk type/structure
                chunk_clean = chunk  # clone/modify as appropriate for your stream object
                if hasattr(chunk_clean, "delta") and hasattr(chunk_clean.delta, "content"):
                    chunk_clean.delta.content = cleaned
                else:
                    chunk_clean = cleaned
                yield chunk_clean
    return process_stream()

Notes • Keep

tool_choice="auto"

(or "required") to allow tool calls. • If your streamed chunk structure differs, adapt how you extract/replace

content

. • If reasoning appears in multiple patterns (e.g., different tags or prefixes like "Thinking:"), add regex patterns for those too. If you want, paste a raw streamed chunk object (or indicate whether you’re using Nova Sonic realtime vs standard Nova) and I’ll give the exact minimal patch you can drop into your agent.