https://www.boundaryml.com logo
Powered by
Heyo, I have been using Claude Code a
# general
y
Heyo, I have been using Claude Code a lot these days and realized some really cool things: 1. BAML makes structured responses a first-class citizen. It seems like Opus encourages the use of structured responses even in scenarios where I didn't think it would help or be possible. Opus seems to really lean on BAML's pros a lot. 2. Opus is pretty good at writing BAML prompts. I haven't written a single line of BAML in this new project. I just need to make sure I explain the objective in enough detail, and Opus goes ahead and figures out the function signatures, prompts, and everything. 3. I've stopped using the playground. This is so shocking. I started using BAML since it shows me the prompt in action, but now I barely use the playground. I just trust that Opus will do a good job with the prompt. - Most importantly, BAML examples are really challenging to write by hand, especially when dealing with multi-turn conversations. - Generating examples using Opus, too, seems pretty pointless. - We really need a way to bring in examples from an external database or something to make sure we are using real-world examples. 4. I have stopped using baml tests as test cases: They are just a way for me to see my prompts formatting and quickly try something out in the playground. My test cases are faily subjective and require the use of an LLM as a judge.
a
Have you tried to make it use baml cli test?
We definitely want to add an easy way to query prod data and iterate on that as the test case
y
Ooo. This reminds me of somehting super important. 4. I have stopped using baml tests as test cases: They are just a way for me to see my prompts formatting and quickly try something out in the playground Most of my tests are extremely subjective (i need to use an LLM as a judge). Writing assets and deterministic test cases doesn't help a lot in most baml functions.
v
I think I can share a case where I used Codex to write prompts, and it can render the actual prompts using the BAML client code without needing to open the BAML playground
y
So I have an explicit message in my baml skill to not run baml tests
v
It is very helpful for me when using typebuilder to inject dynamic prompts for debugging How to Render Complete Requests Offline for Each Turn (Without Making Network Calls) You can use BamlRuntime.buildRequestSync to directly get the final OpenAI request locally (including system prompt text), without needing a model API Key or network:
Copy code
const { BamlRuntime } = require('@boundaryml/baml')

  const env = Object.fromEntries(
    Object.entries(process.env).filter(([, v]) => typeof v === 'string')
  )
  const rt = BamlRuntime.fromDirectory('baml_src', env)
  const ctx = rt.createContextManager()

  const req = rt.buildRequestSync(
    'Orchestrator',
    { messages, current_time: '2025-12-18 10:00' },
    ctx,
    tb ?? null, // Key: whether to pass TypeBuilder
    null,
    false,
    env
  )

  const body = req.body.json()
  console.log(body.messages[0].role) // system
  console.log(body.messages[0].content) // complete system prompt text (including ctx.output_format)
p
A lot of these points make sense, considering how good these coding agents have gotten. I think some sort of custom "adapters" that let the user choose where they let their tests/examples for optimization reside would make it very flexible and usable over time. Some folks like to keep their stuff in SQLite, others in Postgres, and still others in static files. I suppose the static file (json?) approach would make sense as a starting point
Is the BAML skill you have for Claude public @YourTechBud
Would love to see
y
Or the ability to provide a host function which allows me to do things: - Populate test cases dynamically - Run custom assertions or test logic for each test case
The repo is private for now. But I can share the contents. Nothing too fancy:
Copy code
### BAML Integration

- **Source files**: `baml_src/*.baml` (edit these)
- **Generated client**: `src/lib/baml_client/` (DO NOT EDIT - auto-generated)
- Run `pnpm build:baml` after modifying BAML files
- **For BAML documentation**: Use Context7 with library ID `/boundaryml/baml`

### BAML Development Rules

**Function Signature Pattern:**

- Use flat Input class: `{FunctionName}Input` - contains all data needed for the function
- Use Options class for modifiers: `{FunctionName}Options` - contains boolean flags/settings that modify behavior (e.g., `isFollowUp`)
- Signature: `function Foo(input: FooInput, options: FooOptions) -> FooResponse`
- NO nested context objects - flatten everything into Input

**Naming Conventions:**

- Input/Response/Options classes: `{FunctionName}Input`, `{FunctionName}Response`, `{FunctionName}Options`
- Action classes: `{Action}Action` (e.g., `SpeakAction`, `WriteCodeAction`)
- Use camelCase for fields
- Use SCREAMING_SNAKE_CASE for XML tags in prompts

**Code Organization:**

- Shared types go in `narrative.baml` (speakers, summaries) or `actions.baml` (action types)
- Shared prompt logic goes in `common.baml` as `template_string`
- Before creating a new type, check if one already exists
- Extract to `template_string` when prompt logic appears in 2+ functions
- Template strings should take a `header` parameter instead of hardcoding rule numbers

**Tests:**

- Test files live beside source files: `foo.baml` → `foo.test.baml`
- Tests MUST match actual input schema - verify field names and structure match
- Update tests whenever you modify function signatures or input types
- Run `pnpm build:baml` after changes to verify compilation
- Never run the baml test command. We don't like it.

**XML Tags in Prompts:**

- Use plural for collections: `<SPEAKERS>`, `<PREVIOUS_ACTIONS>`, `<COMPLETED_UNITS>`
- Use singular for single items: `<DIRECTIVE>`, `<QUESTION>`, `<CURRENT_UNIT>`
- Be consistent across files - same concept should use same tag name
And I have a separate skill for - brainstorming: helps me figure out what I should put in prompts among other things - Gemini prompting guide
p
Wait how does it learn baml syntax with this?
y
Oh. Context7
p
Ohh nooooo. Context explosion 💥
Token explosion I mean lol
I thought one could write a skill to have the model learn BAML on the fly? Isn't that the whole promise of skills?
y
Its just 1.5k tokens. And I only have two libaries on there. There is a baml skill but thats way too big: https://gist.github.com/aaronvg/75596a0063588440d47f5db1361c8a5f
p
Ah ok
Missed seeing Aaron's file
So there's that and context7 as options, got it
y
I am experimenting with allowing opencode to just search the web to replace context7. It aint bad. But opus barely uses context7. Its gets baml syntax right everytime somehow
YOu can also just give it the link to the llms.txt file too: https://docs.boundaryml.com/llms.txt
p
My experience with context7 on other docs has been atrocious. But maybe BAML syntax is concise enough and the docs well written enough that it's gets it?
y
No clue. I am not too happy with context7 myself. Trying to experiment different ways
p
But context7 has gone through a major architecture overhaul lately. It's much better now I think
Gotta revisit
Kinda explains why you don't hate it 😆
2 Views
PreviousNext
Powered by