https://www.boundaryml.com logo
Powered by
Want ctx.output_format different from schema
# help
j
I'm wondering if I can 'manually' construct an output format reusing some of the mappings that generate the not-quite-json from the BAML type definitions. I have two use-cases: One, I am returning string | DynamicContainer as a means to have an agent that can respond with a normal text response to the human or a tool call. That renders as:
Copy code
Answer in JSON using any of these schemas:
    string or {
      data: {
        // Read the contents of a file.
        tool_name: "read_file",
        // Path to the file to read
        file_path: string,
      } or ...
(the DynamicContainer has a single field and the field type is a union). A dumb model returns {"string": "your answer is ..."} and the function returns this string that happens to contain json, not "your answer is ...". I was thinking of "manually" constructing a prompt:
Copy code
Answer with a string or a JSON object with this schema:
{
  data: { ...
  } or { ...
  } or ...
}
Ok. That is one goal. The other idea is apparently some Anthropic tool APIs are defined implicitly where just by specifying the tool name, the model knows the arguments and argument types etc. so if I have a matching return type to catch the tool call, it should work, and I could omit most of the tool definition to save tokens, and I would presume the model is good at knowing when to call the tool. Here again I would be customizing the output format request. But if I am mixing built-in Anthropic functions and my own functions, I can't use ctx.output_format because it will fully specify all the tool APIs. I am thinking I can construct the output format section of the prompt outside the BAML call (in Python) and pass it in, and just render it like any other variable. But if I have variable unions of types or deeply nested structures I can't have fixed strings and I need a way to generate fragments of the output_format from Python. Does this make sense? Are these facilities available?
g
Hi! I didn't quite understand the first part of your question, but if you can rephrase it, I might be able to help. As for Anthropic tool calling - great point. We are working on support for native provider tool-calling APIs within BAML, because, as you pointed out, our generic take on tool calling doesn't take advantage of some of the provider-specific things. We aim to support that in February.
n
modular api should be the answer. Just take over the control of the context logic yourself
You can take over any piece of BAML's workflow. It is just the tradeoff of using baml vs switch to more imperative solution
j
I guess I am trying to figure out how to customize the rendering of the output format. Phrasing the prompt slightly differently might yield better results, but I don't even know what this would look like. I'm looking into modular API and I think this is what I need. I am also fighting a model that has its own conventions for tool calling and the prompt triggers a tool call response that's not a json object within the response content but some other response format with tool fields. So I should be able to use the pieces and get it going.
Just FYI this is an example of what I am trying to work around (OpenRouter/gpt-oss-20b):
Copy code
---PROMPT---
system: Answer in with a plain string or to perform a tool call, return a JSON object obeying this schema:
{
    data: {
        // Read the contents of a file.
        tool_name: "read_file",
        // Path to the file to read
        file_path: string,
    } or {
        // Write content to a file.
        tool_name: "write_file",
        // Path to the file to write
        file_path: string,
        // Content to write to the file
        content: string,
    } or {
        // List the contents of a directory.
        tool_name: "list_directory",
        // Path to the directory to list
        directory_path: string,
    } or {
        // Execute a shell command and return its output.
        tool_name: "run_command",
        // Command to execute (shell command string)
        command: string,
    },
}
user: there is a file in the current folder where the filename is a persons name. within that file is a number. what is that number?

---LLM REPLY---
<|start|>assistant<|channel|>commentary to=tool.run_command <|constrain|>json<|message|>{"command":"ls -1"}<|call|>
---Parsed Response (string)---
"{command: ls -1}"
v
since the output format is very closely tied to behaviour in the parser, as of right now we haven't made it very modular.
i can imagine we do eventually, but for now the best way to own that would be manually owning the string!
PreviousNext
Powered by