https://www.boundaryml.com logo
Powered by
🟡 Advice on Changing to BAML for Streaming Agents...
# questions
e
I need recomendations about my task: Firstly We are using LangChain in our project and our graph and agenst desingned via LC but now we decided that change agenst with BAML , ı could changed normal agents which are no-streaming but one of them use straming mode , i cannot change with baml becauase in process with normal langchain mode ,it invokes non-straming llm invoke but in graphs its interupted and its converted to straming mode and each chunk taken and forwarded but when i try for baml this process because i dont wanna ruin present architecture , its looking difficult can you give me advice ?
v
I'm sadly unsure how streaming works in langchain, I'll have to look it up!
i would say a better way to ask AI is, can langchain stream a custom python function
Ok looked into it more, and sadly they’re fundamentally different APIs, and they don’t line up. LangChain assumes an OpenAI-style interface, where the model streams back: * incremental message deltas * in a ChatMessageChunk format it already understands * and LangGraph can intercept those chunks and replay them as “stream_mode=messages”. Because of that assumption, LangChain can say: > “Give me an OpenAI-style response, and I’ll handle the token-by-token streaming for you.” BAML does not work that way. BAML streams: * typed partial objects (your own user-defined type) * that are already valid JSON objects *and a final fully-typed, validated blob at the end So you never have to manually stitch JSON together or rebuild the final object. This is great for type safety — but LangChain has no concept of: * a domain object “partial” * merging partials * typed-schema streams * or turning these into chat deltas So when LangChain tries to treat BAML’s async generator as if it were an OpenAI chat model, it breaks. It expects message chunks (
AIMessageChunk
), but BAML is giving typed partials (
ProfessorConsultationPartial{…}
etc.). In other words: LangChain: “I know how to stream words/tokens/messages.” BAML: “I know how to stream typed objects and give you a correct final result.” Both are valid, just totally different abstractions. That’s why plugging BAML streaming directly into LangChain’s automatic streaming layer doesn’t work today. You can still use BAML streaming, but LangChain can’t auto-intercept and transform it the way it does for OpenAI.
e
@Vaibhav (BAML 🐑) Thanks a lot for your advice and support. I have one more question. Right now, we’re invoking the LLM through LangChain in non-streaming mode, and LangGraph interrupts it with stream_mode="messages", as you know. Now I’m considering switching that invocation to BAML’s async streaming instead. My idea is: 1. Use BAML async streaming 2. Receive the typed partial JSON chunks 3. Convert those chunks into a format that LangGraph can understand and emit as stream messages Do you think this approach makes sense? Or would you recommend fully bypassing LangGraph’s streaming layer and using pure BAML streaming end-to-end?
v
Honestly I suspect you’ll end up deleting way more code by removing the lang graph layer and have a much simpler code base. But I would just put a coding agent on it and see what the final code looks like!
If it’s not good, then the first approach will work as well!
PreviousNext
Powered by