QQ regarding streaming tokens It seems we can't tell whether the tokens streamed are a thought that accompany an action (because the think tokens appear as a normal message until the tool call tokens are generated?) Could someone give me a quick rundown of

ThinkAction

vs thought accompanying a tool vs reasoning content vs thinking content/blocks

Quick rundown below. Mostly ad-hoc from my knowledge, someone please correct me if you see anything wrong 1.

ThinkAction

= a tool that the LLM can call when it feels like it a. it’s a bit old tool, but occasionally used; sometimes the LLM is confused / conflicted and wants a scratchpad of sorts; then we might see it use this tool to just dump raw reasoning b. the LLM sends the “thought” here as an argument of the tool call itself, e.g. same way it’d send file name to an open file tool, this one is a think tool with a thought arg c. totally separate from the rest of meanings of thought 😅 2.

thought

accompanying a tool call is actually sent from the LLM as

content

a. source code b. used to be sent as thought too in the past but… umm I think that’s gone c. so it’s the LLM sending a toolcall with some extra text (e.g. “OK I open the file : tool call open_file…” = the thought becomes “OK I open the file”) 3.

reasoning_content

comes from litellm as reasoning_content indeed a. I'd kinda say it’s the “standard” reasoning of an LLM - as much as we can speak of a standard in the wild west of reasoning LLMs b. was initiated by OpenAI models but now they’re doing 5 4.

thinking_block

array is/was the anthropic version of 3, basically, some other LLMs have picked it up too, even Gemini a. corollary: we will probably not see 3 and 4 in the same time and/or in the same LLMs (fingers crossed) 5. there’s also `reasoning_content_item``, which is the “new” OpenAI version of 3, but that doesn’t exist on completions API so I mention it just for the sake of completeness - we can ignore it on completions

tysm! im implementing token streaming for acp in cli rn - will ping you for a review or more questions this helps a bunch