https://www.boundaryml.com logo
@check / assert
# general
a
General poll -- If you've used @asserts and @check in BAML, let us know if you've found them useful. Especially @check! We would like to hear from you.
-
j
Mainly use them to ensure a string.. especially helpful if using symbol tuning Been helpful for ranges of number (for instance a risk score from 1-20) … LLM love outputting 0 or 21 for some crazy reason
v
I have a design that depends on it, but since @check doesn't support dynamic injection, I've temporarily put it on hold. You can check if the new baml language offers better support.
Always need to confirm: Use @@check class-level assertion with the condition set to true class DeleteTask { task_keyword string task_id string? @@check(approval_required, {{ true }}) // Always trigger approval } All operations requiring user confirmation (HITL) are defined in the BAML schema, and the execution layer only needs to check the results of checks prefixed with
approval_*
a
thanks for the feedback!
h
Hvae used them, and think they're great. Asserting tha the LLM will output a specific category for example is good in classification tasks. My only issues are the syntax being unintuitive, especially around asserting that the correct type was returned. Eg. I have a function that returns "Resume | AskForClarification" I kind of want to have a test that goes @@assert(this == Resume) but I think I had to use strings? I don't remember. I asked the baml website AI to help (which was very helpful).
3 Views