https://www.boundaryml.com logo
Powered by
🟡 Did someone say native Rust SDK 🙈🙊🙋🏻‍♂️😁
# questions
j
Did someone say native Rust SDK 🙈🙊🙋🏻‍♂️😁
Don’t see any docs yet but taking a look at the code and I’m excited. Was just about to build BAML into a rust backend we have and was gonna do a rest client.. Can I be a test dummy for this? We have an Axum API with a HTMX front end
v
It’s live and it works 🙂
Cargo install baml
Add rust generator
And it’ll just work
Docs will be live in a bit. We’re doing some stuff to make it more friendly first
j
Slick! I’ll give it a go this morning
Cool stuff got it work on my end! ... Had some trouble with my config and getting the playground to play nice but was just using the wrong URL
v
oh what config?
we should patch that!
j
Its all in refernce to how base urls get handled .. Some providers have redirects properly So in my case LM Studio give you localhost:1234 .. as the base url, but in reality the OpenAI API compatibile endpoints are /v1/chat/completions .. or responses So BAML needs localhost:1234/v1 (Not the worst, I've been adding an error handler in my stack because if 
BASE_URL/chat/completions
doesnt work and there is no 
/v*
in the BASE_URL the next guess is to append that and try again. So slighty user error, slightly provider redirect configuration not handling this for me.. BAML i think is the last one to blame 😛
Just debugging this took longer then I should have spent. The logs in LM Studio show real quick.. /chat/completions is not a valid route.. However BAML playground int he extension was showing an error about an ENV not being valid -- triggering off the endpoint not working ^^ Maybe thats an area of improvement? Some verbose mode where the actaully HTTP request from the extension to the Provider can be observed (to see returned error messages at least)
But actual Rust client generation worked like a charm!
@Vaibhav (BAML 🐑) Some general feedback after using the Rust SDK for all of 3 days - Generated Types dont implement Serialize or Deserialize .. > Option A: Generate #[derive(Serialize, Deserialize)] on all BAML types > Option B: Provide a public serialize_baml_value() method on BamlRuntime or BamlValue > Option C: Expose HostValue publicly so users can implement their own serialization ^^ Maybe I missed something? - The runtime module is private, so if there was a private API for above.. cant access So maybe we could add a feature flag to generate types with Serialize/Deserialize similar to serde crates Im hitting this workflow and its clunky right now 1. Call BAML function: B.AnalyzeText.call(input).await? → returns Analysis 2. Serialize to JSON string: serde_json::to_string(&analysis)? → Currently fails 3. Return in API (axum) response: Json(response) → Requires manual wrapper
v
thanks for all the feedback! re the 
v1
part, yes we def need to make that more debuggable. i think we just need better exception handling. that's a core BAML issue not a rust one, so it'll be happy for everyone when we fix this. 
serde
is def just missing! I totally goofed and missed this 🥲
we do customer fridays every week, so i'll take serde as mine this friday!
but sadly that means you'll have to wait until next week's release
j
No worries! Not using it on anything in production yet and only have 2 prompts so building a little wrapper wasn’t too much of lift. Other than that everything is working awesome!
v
thats great!!! 😄
m
do we feel like the rust generator is in a good spot? I think I want to use it for a hackathon 🙂 otherwise i'll use go
j
I mean I’ve been using it all week and no complaints so send it!
8 Views
PreviousNext
Powered by