<@U0A71G31CDV> Tell me about Query Queue v2 - and ...
# questions-and-troubleshooting
r
@Rocky Tell me about Query Queue v2 - and how to guess the values, if my query queue v1 values look like this:
Copy code
MySQL [(none)]> SHOW VARIABLES LIKE 'query_queue%';
+----------------------------------------------+-------+
| Variable_name                                | Value |
+----------------------------------------------+-------+
| query_queue_concurrency_limit                | 240   |
| query_queue_cpu_used_permille_limit          | 0     |
| query_queue_driver_high_water                | -1    |
| query_queue_driver_low_water                 | -1    |
| query_queue_fresh_resource_usage_interval_ms | 5000  |
| query_queue_max_queued_queries               | 1024  |
| query_queue_mem_used_pct_limit               | 0.85  |
| query_queue_pending_timeout_second           | 300   |
+----------------------------------------------+-------+
r
Query Queue v2 (introduced in v3.3.4 and default from v4.1 onwards) is a slot-based scheduler. Unlike the "v1" queue which primarily counts the number of concurrent queries or checks raw resource thresholds, v2 translates cluster resources into "slots" and estimates how many slots a query needs based on its complexity. Mapping your v1 values to v2 In v2, the primary limit is the Total Number of Slots in the cluster. 1. Concurrency Mapping * v1 Variable:
query_queue_concurrency_limit = 240
* v2 Equivalent:
query_queue_v2_concurrency_level
* How to guess: In v2, the total slots available is calculated as:
Total Slots = query_queue_v2_concurrency_level * (Total CPU Cores in Cluster)
If you want to maintain a capacity of roughly 240 concurrent queries (assuming each query takes 1 slot), you should set the level based on your cluster size: * Formula:
query_queue_v2_concurrency_level = 240 / (Number of BEs * Cores per BE)
_ _Example:* If you have 3 BEs with 40 cores each (120 cores total), set
query_queue_v2_concurrency_level = 2
. 2. Queue Management * v1 Variables:
query_queue_max_queued_queries = 1024
and
query_queue_pending_timeout_second = 300
* v2 Equivalent: Identical. * How to guess: These variables continue to function the same way in v2. They control the capacity and timeout of the "waiting room" regardless of whether the limit is triggered by v1 counts or v2 slots. You can keep these values as-is. 3. Resource Thresholds * v1 Variable:
query_queue_mem_used_pct_limit = 0.85
* v2 Logic: In v2, memory is implicitly managed by dividing the BE's memory into slots:
Memory per Slot = BE Memory / (Cores per BE * query_queue_v2_concurrency_level)
The scheduler ensures queries don't over-subscribe these virtual memory buckets. While the v1 variable
query_queue_mem_used_pct_limit
still exists, v2's slot management is designed to prevent reaching that limit in the first place by being proactive during scheduling. New v2 Variables to Consider If you enable
enable_query_queue_v2 = true
, you should also look at these to tune how "heavy" a query is perceived: *
query_queue_v2_cpu_costs_per_slot
(Default: 1,000,000,000): This uses the planner's estimated CPU cost. Increase this value to make queries take fewer slots (allowing more concurrency for complex queries). *
query_queue_v2_num_rows_per_slot
(Default: 4,096): If a query scans many rows, it takes more slots. Increase this if you want large scans to be less "penalized" by the scheduler. *
query_queue_v2_schedule_strategy
: Default is
SWRR
(Fair sharing). You can change this to
SJF
(Short Job First) if you want to prioritize small, fast queries over long-running ones. Summary of Recommended "Guess" To mirror your current setup while moving to v2: 1. Enable v2:
SET GLOBAL enable_query_queue_v2 = true;
2. Adjust slots: Set
query_queue_v2_concurrency_level
so that
Level * Total Cores ≈ 240
. 3. Keep queue limits: Leave
query_queue_max_queued_queries
and
query_queue_pending_timeout_second
at their current values. Referencesadministration/management/resource_management/query_queues.mdfaq/resource_isolation_faq.mdsql-reference/System_variable.md