@Rocky I see even starrocks 4.1.4 is available in docker hub but we still don't see release for 4.1.3 on GitHub and website are going to skip 4.1.3?

As of August 2026, it is common to see newer version tags appear on Docker Hub before the corresponding release notes or official download links are finalized on the website and GitHub. Based on current community activity and project behavior, here is the likely status of version 4.1.3 and 4.1.4: *1. Was 4.1.3 skipped?* It is highly probable that 4.1.3 was superseded by 4.1.4 shortly after its build. In the StarRocks release cycle, if a critical bug or regression is discovered in a release candidate or a finalized build (like 4.1.3) just as it is being pushed to distribution channels, the team may immediately issue 4.1.4 to include the fix. * Evidence: Users in the community have reported using builds labeled

4.1.3-8a8e186

, suggesting the version existed as a specific git hash/commit. * Tagging logic: If the Docker Hub automated pipeline pushes a tag for 4.1.3 but the release is "stopped" due to a critical issue, 4.1.4 becomes the official stable patch for that window. *2. Why is the website behind?* The StarRocks website and official "Download" page are usually updated manually by the documentation and marketing teams. * Synchronization Delay: Docker Hub images are often the first artifacts to go live because they are tied to automated CI/CD pipelines. * GitHub Releases: GitHub "Release" entries require a manual summary of bug fixes and changes. If 4.1.4 was a "hotfix" for 4.1.3, the team might still be consolidating the changelog for both versions into a single release post for 4.1.4. Recommendations * Wait for the Changelog: If you are planning a production upgrade, it is safer to wait until the GitHub Release or StarRocks Documentation page for 4.1.4 is published. This ensures you know exactly which critical bugs were addressed (and why 4.1.3 might have been bypassed). * Check the

.status

file: You can often see the current development and maintenance status of a branch by checking the

.github/.status

file in the StarRocks repository. * Slack Community: For the most immediate confirmation, the

#release-news

or

#general

channels in the StarRocks Slack are where maintainers typically announce why a specific patch version was skipped or replaced. Current Verdict: You aren't imagining it—4.1.4 likely replaced 4.1.3 to fix a late-breaking issue found during the 4.1.3 build process. Follow the GitHub Releases page for the official announcement.

@Rocky Hi, I may have found a serious issue in StarRocks 4.1.2. In a 3-FE / 6-BE cluster, automatic

_statistics_.predicate_columns

maintenance queries remain running with zero rows scanned and almost no CPU usage. While they are stuck, all BE pipeline scan workers become occupied, and normal SELECT queries cannot start scanning—they also remain at zero rows.

KILL QUERY

does not release the scan tasks immediately, and the FE daemon recreates the internal maintenance queries afterward. Is there a known issue or a safe way to fully disable predicate-column maintenance and recover the scan workers without restarting all BEs?

It appears you have encountered a known edge-case where the internal maintenance tasks for Predicate Column statistics (introduced in v3.5.x and enhanced in v4.x) can enter a stuck state, particularly when the system table

_statistics_.predicate_columns

faces contention or high metadata overhead. In StarRocks 4.1.x, the pipeline engine manages these as internal queries. If the BE pipeline scan workers are fully occupied and do not release after a

KILL QUERY

, it indicates that the tasks are likely blocked on an uninterruptible I/O wait or a metadata lock within the BE's scan context. 1. Safe Way to Disable Predicate-Column Maintenance To stop the FE from generating these queries and to prevent the daemon from recreating them, apply the following FE configurations. You should set these globally to ensure all FE nodes stop the collection immediately:

sql
-- 1. Disable the collection of predicate columns during query optimization
ADMIN SET FRONTEND CONFIG ("enable_predicate_columns_collection" = "false");

-- 2. Disable the persistence task that writes these stats to the _statistics_ table
ADMIN SET FRONTEND CONFIG ("statistic_predicate_columns_persist_interval_sec" = "0");

-- 3. (Optional) Disable automatic statistics collection entirely if the cluster is unstable
ADMIN SET FRONTEND CONFIG ("enable_statistic_collect" = "false");

2. How to Recover Scan Workers Without Restarting BEs If

KILL QUERY

has been issued but the BE scan threads remain occupied (visible as 0% CPU but "Running" in

SHOW PROC '/current_queries'

), the scan tasks are likely orphaned in the BE pipeline pool. * Step A: Verify the Task ID: Use

SHOW RUNNING QUERIES

or

SHOW PROC '/global_current_queries'

to find the

QueryId

of the stuck internal queries. * Step B: Use `KILL ANALYZE`: Sometimes internal stats tasks respond better to

KILL ANALYZE <id>

(if they were triggered as part of an analyze job) rather than a standard

KILL QUERY

. * Step C: Force Pipeline Task Expiry (Experimental): If the threads are stuck because they are waiting for a scan context that no longer exists, you can try to reduce the

pipeline_scan_thread_pool_queue_size

(though this is not mutable in all versions) or increase the scan parallelism temporarily to "push" new queries through, but this is rarely successful if the workers are truly deadlocked. Note: If the workers do not clear within 5–10 minutes after disabling the FE configs and killing the queries, it typically indicates a thread leak or a deadlock in the BE scan connector (often related to system table I/O). In this specific state, a rolling restart of the BEs is the only guaranteed way to clear the blocked pipeline drivers. 3. Why This Happens in 4.1.2 In StarRocks 4.1.x, the system attempts to be more "adaptive" with layout and statistics. If your cluster has a very high number of unique predicates or a large number of tablets in the

_statistics_

database, the internal

INSERT INTO ... SELECT ...

used to refresh

predicate_columns

can hang if the BEs are under high load or if there is a version-compaction backlog on the system tables. Recommendation: After disabling the features, it is highly recommended to upgrade to 4.1.4+ once available, as these versions include specific fixes for internal metadata query stability and scan worker lifecycle management. If the issue persists after a BE restart and disabling the configs, please capture a

jstack

from a BE and the Lead FE to check for lock contention on-going lock contention in the

ShardManager

or

TabletInvertedIndex

. References • knowledge/trouble_shooting/fe_deadlock_problems.md • knowledge/admin_and_operation/shared_data_compaction.md