Hi! Is there a function to "flush" all stream left...
# agents
a
Hi! Is there a function to "flush" all stream left from the llm? I'm trying to make it so when the llm respond with some specific text, we reprocess the whole llm call, but if the text is found mid sentence the previous stream is concatenated at the beggining. Any ideas?
b
Hmm not sure if there is a flush method. But an alternative idea might be to wait for the full llm answer instead of processing it chunk by chunk of the stream. Then you can process the whole line as you want. You might want to try to use something like groq for fast token generation if you go that route.