I see that 4.1.5 was skipped for dockerhub but I see 4.1.6 published as of 4 hours ago. I am hopeful that this version improves some of the instability we have with shared-data architecture, specifically primary key tables. @Rocky can you outline some of the impactful changes for shared-data users going from 4.1.1 to 4.1.6?

Based on recent StarRocks releases and development in the 4.1.x branch, here are the impactful changes and stability improvements for shared-data (cloud-native) architecture and Primary Key tables moving from 4.1.1 toward the newly released 4.1.6. 1. Data Correctness and Duplicate Row Prevention One of the most critical fixes for Primary Key tables in shared-data clusters involves the handling of delete vectors. * Delete Vector Application: A fix was introduced (carried through 4.1.3+) to ensure that the "publish" phase in shared-data Primary Key tables correctly applies per-segment delete vectors. Previously, a race or metadata mismatch could lead to "duplicate" primary key rows remaining live in the system. * GIN Index Correctness: For users using inverted indexes (GIN) on Primary Key tables, a fix ensures that the index reads from the correct segment after a column-mode partial update, preventing the service of stale index data. 2. Shared-Data Metadata & Transaction Stability The shared-data architecture relies heavily on the reliability of tablet metadata stored in object storage. * Checksum Protection: Added checksum protection for shared-data tablet metadata and transaction logs. This helps detect and prevent corruption that could occur due to transient object storage issues. * Pending Publish Visibility: A new FE metric was added to track the maximum pending-publish time of committed transactions. This is a vital diagnostic tool for shared-data users to identify why data visibility might be lagging even after successful commits. * Schema Change Locking: Scoped shared-data schema-change job locks to the table level. This significantly reduces lock contention across the cluster when performing schema updates on different tables. 3. Storage and Resource Management (GC Improvements) A known issue in earlier 4.1 versions caused shared-data Primary Key tables to accumulate unreferenced persistent-index SST files in object storage, leading to "bloated" storage costs and potential performance degradation. * Persistent Index Garbage Collection: Refinements were made to the garbage collection (GC) and thread eviction mechanisms for handle persistent indexes. Version 4.1.6 (and the late 4.1.x patches) includes fixes to ensure obsolete index data is properly reaped from object storage (#78236). 4. Why 4.1.5 was likely skipped In the StarRocks release cycle, a patch version is occasionally "skipped" for public distribution (like DockerHub) if a critical bug or build failure is discovered during the final validation of that specific tag. Version 4.1.6 effectively serves as the "cumulative" patch that includes the fixes intended for 4.1.5 plus any urgent corrections discovered during that final validation. Summary Checklist for Shared-Data PK Users: * Check `enable_persistent_index`: From 3.3/4.0+,

CLOUD_NATIVE

is the recommended persistent index type for shared-data to avoid local disk dependency. * Monitor Tablet Metadata: Utilize the new checksums and FE metrics to ensure your object storage backend is performing reliably. * Verification: After upgrading to 4.1.6, monitor your S3/Object storage bucket for a reduction in "orphan" SST files if you were previously experiencing storage bloat. For the full detailed commit log, you can refer to the StarRocks GitHub Releases page. References • release_notes/release-3.3.md • deployment/shared_data/feature-support-shared-data.md • release_notes/release-3.2.md • release_notes/release-3.5.md