https://www.boundaryml.com logo
Powered by
🟡 Support for loading data in BAML tests
# questions
r
I want support for loading data in BAML tests. It’s a simple feature.
They quickly grow to make the actual model file too big. Split it up.
g
Soon we'll have the ability to define big strings in 
template_string
and use those strings in tests. Would that solve your problem with the model file becoming too large? We also have a new cli flag that strips test code out of your generated baml_client: 
baml-cli generate --no-tests
. In case it's useful. What would the ideal data-loading interface be, for you?
r
The first feature for loading documents is what I need
g
Ok. Something like this that works just in tests?
Copy code
test Foo {
  functions [Bar]
  args {
    my_long_string  File(../fixtures/my_file.txt)
  }
}
Or more like a format for specifying tests themselves?
Copy code
$ cat my_tests.csv

function | args
Foo | { my_long_string: "lalala" }

$ baml-cli test --input-data my_tests.csv
We are considering something like the latter, to improve prompt opmitization?
(BTW did you see the prompt optimization announcement? I thought you might find that interesting. In fact it was one of your tweets that made me suggest to the team that we finally bite the bullet and implement that!)
r
The latter would be good, minimal is fine too
What changes am I hearing about for building knowledge graphs?
I did see the BAML one, thanks
g
Cool, thanks for the feedback. I'm not sure what changes you're hearing re: knowledge graphs, but if you link me to a comment I might be able to find the context.
r
I just heard a rumor it's gonna get easier to do directly. It wasn't specfic. I want to know your GRAND PLANS 😄
p
This is the sort of thing I was expecting to see in template strings, having a lot of modular prompts. e.g. converting a 200-line recursive condenser, but being able to break-up and re-use components. Had been putting my template strings in separate BAML files but the support for csv there probably solves my use case as well. Cleaner than using environment variables probably. It’s good to keep things simple and expandable
2 Views
PreviousNext
Powered by