How can Prefix Caching be implemented in BAML's agent? I am currently using a dynamic schema where tools are injected dynamically, but this causes the system prompt to be disrupted, leading to caching failure.

Caching is implemented on the model side and not by BAML. Unless you are using an on-premise/Your-Own-Custom-Model , there is nothing to be done from BAML side. There are some ways to address it though: - Keep your static content at the top of the prompt - Keep the dynamic content at the end of the prompt. In your case, the tools if it keeps changing should be at the bottom of the prompt. Sort the tools list before you inject it into the prompt, so it is not random all the time That way you will get cache hit even though the tools keep changing

Yes, I understand what you're saying. I know this is an implementation on the model side. I'm just thinking about how to use BAML to achieve best practices for caching. Thank you for your reply.

Ya...there is no option like

ctx.tools

Here is a video by boundary that goes deep into the topic https://github.com/ai-that-works/ai-that-works/tree/main/2025-05-27-mcp-with-10000-tools You can create tools dynamically using type builder and select the appropriate tool using various method like embedding etc, not just LLM to narrow your selection. The prompt could be something like below:

function PickAction(state: string) -> Actions | HumanMessage {
  client "openai/gpt-4o"
  prompt #"

    Help the user by picking an action for the following.

    {{ _.role('user') }}
    {{ state }}

You are an agent with access to any number of tools.

    {{ ctx.output_format }}
  "#
}

Hope this helps

Yes, I have watched all the videos in the "AI That Work" series.