I’m trying to process a ~10-minute video that contains about 150 UI screens. The goal is to extract structured data from those screens using BAML, basically, feed each frame to a model and build a typed struct. I’ve read [this section](https://docs.boundaryml.com/ref/baml/types#video) of the docs, which suggests that

GPT4Turbo

supports video input. However, when I try it, I get an error saying video input is only supported through OpenAI’s Realtime API. I’ve also tried

gemini-2.5-flash

, which technically works, but the OCR/text extraction isn’t very reliable, for example, it often reads “6” as “8” or drops smaller UI text. So I’m wondering: - Is there currently an OpenAI model supported in BAML that can handle video input directly? - Or should I be extracting frames from the video and treating them as individual images? - And, if frame extraction is the recommended path, are there any best practices for reducing frame rate or feeding many frames into a BAML pipeline efficiently? https://cdn.discordapp.com/attachments/1424218342637436978/1424218342885031986/image.png?ex=68e32631&is=68e1d4b1&hm=4c6b73b1135520d324b72da397074b53b5df6242eb550c55bc3df6b3b7ef30c4& https://cdn.discordapp.com/attachments/1424218342637436978/1424218343921160263/image.png?ex=68e32631&is=68e1d4b1&hm=1eeb8044e86bed9d102d1ff34535b93972ffbbcf2e5a052938c5bba707e9a9ca&

Thanks @veth0119 I’ve tried

openai-responses

but it fails with video as an input

i think they are not really supporting video inputs on openai models 🙁 https://cdn.discordapp.com/attachments/1424218342637436978/1424839386742259843/Screenshot_2025-10-06_at_12.22.49_PM.png?ex=68e56896&is=68e41716&hm=6e75c05c4cb6448dfc2ee78ec90d8f6b4d9bd0e47d1a4481c96921fa393e5cfd&

I built a lil zoom sequence that identifies relevant section and Gemini model is pretty accurate this way 🙂