Hey guys. I know we have addressed this question in the past already but I'd like to revisit this once again. I love BAML's structured response & semantic streaming support. My product is built on top of it. But give how good models are getting in tool calling and constrained generation, do we still see value in having baml as a layer on top? I mean eventually we'll reach a point where models just get unnaturally good at this things (even the smaller models)

Always a good question! My thoughts: A. ever since the first JSON mode, that tension has existed, and we haven’t really seen a degradation in the utility of schema aligned parsing vs. tool calling APIs, even for the big models. With SAP you’re free to mix JSON outputs with whatever other text you want. B. we continue to be a big reliability boost for small models. Empirically we haven’t reached the point where tool use APIs can replace SAP, C. Things you mentioned like semantic streaming are never going to be natively supported in the model, BAML gives you that uniformly for all models. D. Even if small models get good at structured outputs, BAML could give you comparable performance in even smaller models. E. DX, iteration and testing loop Those factors get weighed against the costs of BAML - awkward setup, leaving your host language, version management, deployment complexity. We continue to change the calculus on the costs side, too. I’d be curious about how the pros & cons play out for different projects according to their schema complexity, reliability needs, base model, streaming needs, dev team size etc, maybe there is a clear decision boundary.

I think one big cost to also consider is that as these model APIs keep improving and adding new features, there will always be some strain on the baml team to keep up. For example: Thinking tokens (i knows its supported but a bit unconventional, interleaved thinking, and whatever new stuff these guys throw our way)

The opportunity cost of those specific features, yes.

FYI i do think we should just natively add function calling support into BAML clients natively!

BAML is awesome because it allows bringing that power into smaller models.

Hey @greg (BAML) , I just came across another cost of using BAML: Inability to use Claude Agent SDK. Honestly, I hate the Claude Agent SDK, but it unlocks a whole new business model for me. - It allows me to offload the cost of running inference to the user. - Folks can just use their existing Claude Max subscription. - I believe other providers will follow suit. P.S. I really love BAML and am using it in my own project very heavily. Just trying to share things as I realize them with the sincere hope that it will help.