Hey, I’m trying to “multithread” images to a VLM. ...
# agents
a
Hey, I’m trying to “multithread” images to a VLM. What i’d liketo do: grab a VideoFrame triggered by user starting speech, and immediately send it to a VLM, with a dummy prompt if necessary. Then in on_user_turn_completed, I send the same exact image “again” with the actual user prompt appended this time, but now the VLM has cached the image. Is this achievable?
p
Yes, that’s achievable. You can pre-send the VideoFrame with a placeholder prompt to warm the VLM cache, then reuse the same image reference (or hash) in on_user_turn_completed with the real prompt so the model leverages the cached embedding instead of reprocessing the image. I’ve implemented similar vision+speech pipelines and can help design the caching flow and latency optimization end-to-end. @acceptable-leather-47181