Hi team! @Rocky Quick question regarding the roadmap for Iceberg v3 support. We are currently pushing high-velocity streaming data into Apache Paimon Primary Key tables and exposing them to StarRocks 3.5 Right now, we are hitting a performance wall: 1. StarRocks' Iceberg Catalog doesn't natively parse Iceberg V3 deletion vectors yet, forcing slow row-by-row Merge-on-Read. 2. Switching to the native Paimon Catalog triggers Frontend (FE) JVM OOMs due to the massive volume of un-compacted real-time LSM metadata. I see that initial V3 row-lineage features are landing, but what is the rough timeline for the native C++ Backend (BE) Deletion Vector bitmasking engine and Puffin parser? Is this targeted for an upcoming 3.5.x patch, or is it planned for the next major release (e.g., 4.x)? Any context or open GitHub PRs we can follow would be awesome Thanks! 🙌

Hi there! That is a very specific and timely question. Given the current progress in the StarRocks 4.x cycle (as of mid-2026), here is the context on the Iceberg v3 roadmap and the issues you're seeing: 1. Iceberg v3 & Deletion Vector Roadmap You are correct that the initial Iceberg v3 foundation is landing in StarRocks 4.1. This release includes the schema-level support for new v3 types (specifically the VARIANT type) and the Row Lineage metadata columns (

_row_id

,

_last_updated_sequence_number

) which you've likely seen in recent documentation or PRs. Regarding the Native C++ BE Deletion Vector (DV) engine and Puffin parser: * Timeline: This is a major architectural milestone and is planned for the 4.x series (targeting 4.1 or 4.2) rather than a 3.5.x patch. While 3.5.x continues to receive performance and stability backports, the native v3 DV bitmasking engine requires significant changes to the BE scan nodes to handle the Puffin-based Roaring Bitmaps. * Context: StarRocks already implemented a similar native C++ engine for Delta Lake Deletion Vectors in v3.4.1. The team is leveraging that experience to build the Iceberg v3 equivalent. The goal is to move from the current "slow-path" (Merge-on-Read via JNI/Java SDK) to a fully vectorized C++ implementation that applies the Puffin-stored bitmaps directly in the BE. * GitHub Tracking: You should follow Issue #60956 (Add support for Apache Iceberg V3) and Issue #67632 (StarRocks Roadmap 2026), which tracks the broader "Iceberg Delete Support" and "Real-time storage layout" enhancements. 2. Paimon Catalog FE OOM The FE JVM OOM you're experiencing with the Paimon Catalog is a known side effect of real-time LSM metadata overhead, especially when there are many uncompacted snapshots and manifest files. * Why it happens: The Paimon Catalog in StarRocks currently caches metadata to accelerate query planning. For high-velocity streaming, the volume of small, uncompacted LSM files can exceed the FE's default memory limits during metadata synchronization. * Interim Suggestions: * Metadata Cache Tuning: Check if you can reduce the metadata cache expiration or size. * Paimon Compaction: Ensure your Paimon background compaction (dedicated Flink job) is keeping up. High "un-compacted" volume is usually the root cause of metadata bloat. * 3.5.x Optimizations: StarRocks 3.5.3+ introduced optimizations for Iceberg/Data Lake metadata scans (e.g., shared file I/O for metadata) to reduce FE memory pressure, though Paimon-specific metadata handling is still being refined in 4.x. Summary * 3.5.x: Primarily stability and metadata scanning optimizations. Full v3 DV native engine is unlikely here. * 4.1/4.2: The target for the "fast-path" native BE Deletion Vector bitmasking and Puffin parsing. * Recommendation: If you are hitting a performance wall in 3.5, the StarRocks 4.1 release is the one to watch. In the meantime, increasing Paimon compaction frequency is the best lever for the FE OOM issue. Feel free to ping the

#iceberg

or

#paimon

channels in the StarRocks Slack if you want to get more granular details from the engineers working on the BE scan node optimizations! octocat #66944! 🚀 References • administration/management/BE_configuration.md • using_starrocks/distinct_values/Using_bitmap.md • table_design/indexes/Bitmap_index.md • data_source/feature-support-data-lake-analytics.md

I’m also testing Paimon and Iceberg

Hi @Victor Babenko! we’re still testing both as well and haven’t found a clean solution yet

I got it to work but actually had to fork and patch both SR and Paimon