https://www.boundaryml.com logo
Powered by
🟡 BAML Playground Streaming Issue with `stream fa...
# questions
c
Hey! It seems BAML Playground streams the response even if I've set 
stream false
on the client, is that expected? For context: I was showcasing BAML Playground today, using an in-house LLM proxy as the client, and it started failing with a 408 that related to streaming. The cURL request was working fine, though. As it turns out, our in-house proxy doesn't yet support streaming, which I was taking into account by setting 
stream false
in the client definition. I then tested the same prompt on an OpenAI client with 
stream false
, which seemed to stream the response on BAML Playground.
a
can you try adding 
support_streaming
in the 
options
?
Copy code
options {
  supports_streaming false
}
c
Aaaa it works perfectly, thanks @User !!!
Also, it isn't clear to me why 
openai-generic
is recommended over 
openai
when using OpenAI-compatible proxy. What is the high level reason for separating those two?
I guess it's just the defaults for `base_url`/`api_key`/`supports_streaming` right
a
yep exactly
PreviousNext
Powered by