Hi BAML team! with a simple function like this

function Invoke() -> string {
    client ChatBaml
    prompt #"
        {{ _.role( 'user' ) }}
        {{ 'Hello, how are you?' }}
    "#
}

vllm renders it as ( Hermes format):

<|im_start|>user\nHello, how are you?<|im_end|>\n<|im_start|>assistant\n

Is there currently (or planned) support for controlling / injecting a custom string right at the start of the assistant turn, for example: {{ _.role( 'assistant', prefix='assistant{"name": “browser' ) }} will be rendered as

...<|im_end|>\n<|im_start|>assistant<tool_call>{"name": “browser

This pattern is very useful for reliable tool-calling agents (inspired by the Manus blog post "Context Engineering for AI Agents: Lessons from Building Manus"). Thanks in advance!

we currently onlly really support the openai/anthropic messages! currently the best workaround here is the modular api, but i do agree that it would be nice to have some more control here as a developer!

I watched the BAML video about Manus and also read the Manus blog, and what I’m trying to do now is maximize KV-cache hit rate. The main issue is that the context needs to be 100% token-stable (or at least mostly token-stable). When I use another BAML function for a subset of tools, the system prompt changes, which immediately causes a cache miss. If I instead use another LLM client to prepend an assistant prefix, I lose the benefits of BAML’s parser 😅 So yeah… it feels like a trade-off for now. I’ll probably work around it and wait for future improvements, haha.

oh seems like i duplicated that. I’ll follow the thread now