https://www.boundaryml.com logo
Powered by
Is anyone seeing discrepancies between responses f...
# help
b
I have a prompt that consistently shows a different response with baml vs with the gemini sdk or even through the gemini UI console. Not really sure how to start debugging this, any help would be apprecaited!
tried updating to the latest baml-py but no luck
b
That definitely sounds weird- does the raw curl command help here at all?
a
i can hop on a call
if you'd like. But does the curl request do the same thing?
and youre using the Vertex endpoint, or Google AI?
and by curl request i mean playground > curl tab
b
i was struggling to get the curl to run outside the playground
a
Do they show you the actual curl request there?
You can also look at the network tab in “open webview developer tools” in vscode
When you run something in the playground
It shows the full request
So you use vertex i see
And baml is also using the vertex provider?
b
yea i use vertex and baml also uses vertex
maybe we can jump on a call in 30 min?
a
Works for me
ready whenever
summary: the issue is likely due to some whitespace stripping discrepancy happening in the vertex ui / baml. Causes the model to flip its responses. Gave some tips on how to debug using the network-dev-tools in the playground
PreviousNext
Powered by