I'm trying to implement a BAML chatfunction that will be used in different places, with different tool calling configurations. For example, call site 1 will have no tools available, and thus should output a string. Call site 2 should be set to automatic tool calling mode, where it can either call the tool, or just respond with text. Call site 3 should be set to

required

where we want to force the LLM to call a tool. We also only want to create one chatfunction for these different configurations. What is the best way to do this?

Tool use flags are set via headers, IIRC? If that's the case, then you would want to use [ClientRegistry](https://docs.boundaryml.com/guide/baml-advanced/llm-client-registry) in order to choose at runtime what client config to use at each call site.

I see. So if we make a chatfunction that is set to auto tool calling mode (it can either call the tool or return a string), we should do it as such:

class tool_1{
  a: int
  b: int
}

function MyFunc(input:str) -> string | tool_1

This should also work if we set to required mode, where we would have:

function MyFunc(input:str) -> tool_1

or none mode:

function MyFunc(input:str) -> string

Oh, I see. You want the return type to reflect the tool-calling mode. The easiest way to do this, if it's acceptable for use case, is something like this:

type Tool = tool_1 | tool_2 | tool_3

function MyFunc(input: string) -> Tool[] {
  client WillBeOverriddenByClientRegistry
  prompt #"Choose tools to satisfy the query
    {{ input }}.

    (( ctx.output_format }}  
  "#
}

I want this single chatfunction to support all tool-calling modes (required, auto, none), would i just do this then (assuming that if there are no tools called it would return a string)?

type Tool = tool_1 | tool_2 | tool_3 | string

Let's see. In Required, you'll get 1 tool call and no string. In Auto you'll get maybe a tool call and maybe a string. In None you'll get only a string. It sounds like there is no case where you would want & get multiple tool calls.

class Response {
  tool_call: Tool_1 | Tool_2 | Tool_3
  message: string?
}

This type captures all those possibilities. Also just making sure, you're using the

openai-responses

client type?

yes we're using the openai-responses endpoint for this specific case, but we dont want to overindex on openai usecases, since one of the main points of using BAML for us is that its provider agnostic