“Daft is a distributed query engine that’s going to be replacing Spark.” I remember this talk like it was yesterday 👇 Two years ago @ SF Systems Meetup, Colin Ho and Desmond Cheong walked up and floored the crowd - debuting v1 of Swordfish, our local execution engine. Even at the time, Daft demonstrated dramatically lower memory than Spark on TPCH. 17 months later we've delivered multiple iterations of Swordfish and Flotilla with compounding adoption across top labs and startups. Daft's execution model contrasts directly with Spark: Spark: \- Pull based, Partition Driven \- JVM runtime \- Bulk Synchronous Daft: \- Push-based, morsel-driven streaming. \- Async Rust runtime. \- Dynamic parallelism. Even at the time, the original benchmarks showed dramatic improvements on both memory and speed. It was a seminal moment for the team and the open source community, not just for it's technical depth but it's coherence of vision. Our latest benchmarks: https://docs.daft.ai/en/stable/benchmarks/ Watch the full talk on youtube: