https://www.boundaryml.com logo
Powered by
🟡 Streaming Issue in Next.js Localhost with Perfe...
# questions
r
Hey folks! Noticed a particularly weird issue when streaming using the hook in Next.js. The streaming gets stuck on localhost often. Upon inspection, here's what I found: - 
isLoading
stays true - Data is streaming (checked in network tab), but the UI stops updating - 
onStreamData
isn't called after a couple of times - 
onFinalData
is never called whenever the stream is stuck Here's the weird part: The streaming work PERFECTLY when deployed to Vercel. This only happens on localhost. I've tried increasing timeouts on LLM level, disabling ad-blocker on localhost etc. But unable to figure out the reason behind the discrepancy. Has anyone else faced this? What could be the issue?
Hey @Vaibhav (BAML 🐑), would love for you to see this when you have 5. Sorry, if I'm bothering - no rush.
Oh crap , completely forgot that today is Sunday. My bad. Please take your time.
a
Interesting, I'll take a look. Weird that it works on Vercel though, wonder if it's more of a "works in production bundle" but not debug.
r
Yeah, possibly. I'm only here because it's hurting the local development experience, and because not knowing the reason behind this discrepancy is driving me nuts in the background
a
did some quick research with Claude and it may be due to how we hanlde incoming chunks, I'm not completely sure if this is true though, I'll spend some time see if I can fix it https://cdn.discordapp.com/attachments/1462351633861181526/1462797857097646144/image.png?ex=696f802a&is=696e2eaa&hm=3c65c9a8d8caa6c5415aae57782d297d030997b05600094ae86c1bbcc30e10bf&
r
Hmm, this is very similar to what Cursor found when I let it run free across the project to figure out the issue. Happy to look into this as well. Will keep you posted. Let me know if I can help in any way.
a
Alright, got this [PR](https://github.com/BoundaryML/baml/pull/3000) up, I think this fixes the issue
at least I tested streaming in the browser and it works
also integ tests with hooks work with streaming
r
Oh beautiful, that was quick! Thank you so much for the promptness @Antonio Sarosi (BAML 🐑), really appreciate it 💫
How can I test this? Do I have to wait for the next release?
v
release is going out today! 🙂
r
Love it, thank you folks! Crazy pace
v
sorry for the mixup! likely will be today not tmrw 🥲 tagging @aaronv (BAML) for context
r
Any ETA on this folks? Sorry, don't wanna be a bother. Just wish to plan things accordingly.
a
releasing right this second actually
but it will take like 1 hours-ish to propagate
but you should expect 0.218.0 before tomorrow
r
Awesome, thank you very much!
What timezone are you folks operating in?
v
PST
yes its 11 pm, we just work late 🥲
r
Understandable, take care folks 😭
Can see the release (0.218.0)! Giving it a go soon. Thanks again folks, y'all are awesome! 💫
Also, quick question: how do you cancel stream using the hook?
a
We dont have a way to cancel yet from react, youll have to ignore the stream for now
r
That would have been fine, except I think the stream is blocking the UI. Until the streaming is finished, any 
router.push()
calls are simply put in the queue and executed after the stream ends. It hinders native navigation attempts from the user. Current workaround is using 
window.open()
. This is a separate issue you folks can look into.
Also, hate to that person again, but I think the localhost streaming issue still persists. Let me know if I am missing something.
a
@Antonio Sarosi (BAML 🐑) maybe you can try repro tomorrow for Friday
a
@Rishabh can you give me a minimal repro with the bug?
@Rishabh if you can give me a minimal repository that has the issue so I can look at it would be helpful
r
@Antonio Sarosi (BAML 🐑) Hey! So sorry, I missed these messages. Allow me some time. Let me try to repro this locally and come back with a repository.
PreviousNext
Powered by