Token streaming for chat completion API :slightly_...
# dev-pr-review
c
Token streaming for chat completion API 🙂 (I realize i need this for one of my side projects - when sending really large request to LLM, w/o streaming it is really easy to get litellm timeout) https://github.com/OpenHands/software-agent-sdk/pull/1270
✅ 1
@calm-artist-7206 I took your PR, simplify it a bit, and fork off it to get to this PR 🙂 I pushed to your PR too 😭 sorry about that! (I initially try to get your PR merged first, then i realized the responses API streaming is a bit more complex than i originally envisioned) LMK if you want me to revert those changes 🙏
❤️ 1
c
Oh this is awesome!