Hi Team, Mehul from DigitalOcean's . We're in the middle of evaluating Daft & Raydata for a multimodal ingestion & transformation on Kubernetes (Heterogeneous GPU/CPU, Batch Inference, sinking to Lance + Iceberg with Gravitino catalog registration). Daft is technically the better fit on most dimensions that matter to us, RayData seems to be good too but need alot of wrapper on low level API, but surely winning in corporate marketing with anyscale summit. Before we commit, I'm trying to get a clearer picture on a few things, and the docs only go so far: 1. Real-world production deployments on self-hosted K8s The public reference set (Amazon BDT, CloudKitchens, Together AI, Essential AI) is well-documented but mostly 12–18 months old. Are there newer teams running Daft on self-hosted Kubernetes not Daft Cloud, not Anyscale at meaningful scale? On that note: we're aware that ByteDance's LAS (Lake for AI Service) team has been one of the most active contributors on the Daft↔️Lance integration surface distributed FTS, compaction, schema evolution, limit pushdown, and most recently

GravitinoGvfs

local filesystem writes in v0.7.10 (

@qingfeng-occ

). Their pattern Lance as storage, Ray as runtime, Daft hardening the client layer is close to our target architecture. If anyone from that team or others running a similar stack is in this Slack, I'd love to connect. 2. Flotilla vs set_runner_ray() — current recommendation for DOKS We're open-source-only. Is the Flotilla native K8s path production-ready today for a team that can't fall back to Daft Cloud? Or is the practical answer still

set_runner_ray()

+ KubeRay for anything serious? Any known stability cliffs under sustained load? 3. Checkpointing roadmap for Lance/Iceberg sinks

daft-checkpoint

scaffolding landed in v0.7.6 — great signal. Our write path is Lance + Iceberg + OpenSearch, and the current Parquet-only limitation means we'd still need external idempotency for v1 regardless. Is Lance/Iceberg write support on the near-term checkpoint roadmap, or is that explicitly out of scope for now? We'd plan to contribute back when things fall in place. Happy to share our UC matrix and PoC findings once we're further along.

A few helpful resources: Daft's lance integration has moved to a dedicated extensions repository: https://github.com/daft-engine/daft-lance K8s as a dedicated backend is a 2026 priority for us along with several other items that may be of interest: https://github.com/Eventual-Inc/Daft/discussions/6562#discussion-9799154 For K8s Specifically: https://github.com/Eventual-Inc/Daft/discussions/6639 Checkpointing is in the near term roadmap (eta early June) @Rohit Kulshreshtha Definitely interested in your findings. Sounds like your team would prefer k8's native flotilla over ray?

Iceberg exactly once writes with checkpointing are part of the near term roadmap.

• Awesome good to know that, is there any benifit of using Daft's lance instead of pylance? • I am more inclined towards k8's kuberay operator to run daft to get application/job level isolation, Also I see kuberay more mature, I need the setup hitting the ground in a month, correct me If i am missing anything? • Amazing @Rohit Kulshreshtha is the checkpointing be done for only iceberg and or it will happen for lance aswell in June release?

Right now it's mainly for iceberg. community contributions will follow closely for delta and paimon. Lance supports 2PC through fragments, but there were some open questions on idempotence of the commit operation which we haven't prioritized at the moment.

what would be the best way for the time being to have checkpointing for lance! any rough ideas?

There's no stop gap unfortunately. The checkpointing implementation is essentially a 2PC implementation within daft so the sink has to participate in a first class way for the checkpointing to work with that format. You could do a copy from something that supports checkpointing (iceberg) to lance. Also, the idempotence limitation i mentioned earlier: that's an open question and is a limitation of lance, not of daft - so doing it right may need changes contributed to lance itself. Iceberg and delta do not have this limitation.

Awesome good to know that, is there any benifit of using Daft's lance instead of pylance?

@Mehul Batra, Daft+lance provides many distributed operations on top of lance. Such as distributed writes. daft-lance package also provides functions for doing distributed ops on top of daft, such as creating indices, compaction and more. It's very similar to the lance-ray package.