able-lighter-67088
09/20/2025, 11:14 PMtall-airline-16756
09/20/2025, 11:14 PMtall-airline-16756
09/20/2025, 11:14 PMsession.interrupt() method to gently pause the agent’s speech and switch it back to listening mode. This way, the agent is ready to process and respond to the new input right away.
Here’s a simple way to handle an uploaded image:
1. Use the sendFile method from the LiveKit frontend SDK to send the image on a specific byte stream topic, such as "images."
2. Set up a byte stream handler in the agent's on_enter() method to listen for files coming through the "images" topic.
3. Once an image arrives, call session.interrupt() to pause the agent’s current speech.
4. Convert the image into a base64 data URL and add it to the chat context with an ImageContent object, so the agent can easily refer to it in the next response.
5. After this interruption and context update, the agent will take in the new info and reply accordingly.
This approach helps keep the conversation smooth and responsive, especially when working with multimodal inputs like images. If you have any more questions or need a hand with implementation, I’m here to help!
Sources: Agent speech and audio | LiveKit Docs | Turn detection and interruptions | LiveKit Docs | VoicePipelineAgent | LiveKit Docs | Vision | LiveKit Docs