👏 Our very own @Kevin Wang will be speaking at the Ray Summit: How Daft Optimizes Batch Inference Throughput and Prefix Cache Hits Using Dynamic Partitioning and Streaming Execution Daft is used to run petabyte-scale multimodal queries on Ray, where batch inference is often one part of a complex pipeline. By combining vLLM’s high-performance inference with a novel “dynamic prefix partitioning” strategy, Daft is able to achieve the best performance on large scale batch inference workloads. Traditional approaches for maximizing prefix caching involve a manually specified, full upfront sort and partition of your prompt dataset, during which GPUs sit idle. With dynamic prefix partitioning, Daft eliminates this bottleneck by continuously and automatically adjusting partitions in-flight as data is streamed into vLLM. This allows Daft to simultaneously optimize for prefix cache hits and keep GPU compute saturated throughout the entire lifetime of a query. Join us to discover the inner workings of Daft query optimizer and execution engine, explore performance benchmarks on real multimodal workloads, and learn how Daft can revolutionize your batch inference pipelines. _When_: November 5th (400 430 PM PST) _Address_: Marriot Marquis San Francisco _Room_: Golden Gate C3