Hi guys, @Rocky, we seem to be unable to the the profile data of queries consistently, this happens on a direct connection with a FE node: • we run the query with hint variable as comment with all the variables set to make the query profiled • we grab the last_query_id() as a fallback • we try multiple times to SYNC, get the profile list, look through that for our query id And we still can't seem to find our query in the profile list consistently. And this is through a single connection directly to the FE node

The issue you're describing—profiles not appearing consistently even on a direct FE connection—is often related to how StarRocks handles profile reporting across multiple FE nodes. Here are the most likely reasons and how to fix them: 1. Profile Reporting to Leader FE By default, query profiles are reported only to the Leader FE. If you are connected to a Follower FE, it may not have the profile data in its local memory, even if it was the node that executed the query. * Check: Run

SHOW FRONTENDS

to see which node is the

LEADER

. * Solution: Connect directly to the Leader FE's IP for profiling tasks, or check the Leader's Web UI (

http://<leader_fe_ip>:8030/query

). 2. FE Configuration:

enable_collect_query_detail_info

This FE configuration parameter must be set to

true

for the FE to cache and display query details in the profile list. * Check: Run

ADMIN SHOW FRONTEND CONFIG LIKE "enable_collect_query_detail_info";

. * Solution: If it's

false

, set it to

true

(it is mutable at runtime):

sql
    ADMIN SET FRONTEND CONFIG ("enable_collect_query_detail_info" = "true");

3. Session Variable:

is_report_success

While

enable_profile = true

enables the feature, some versions or configurations require

is_report_success

to be

true

to ensure profiles for successful queries are sent from the BEs to the FE. * Solution: Explicitly set this in your session:

sql
    SET enable_profile = true;
    SET is_report_success = true;

4. Cache Eviction (

query_detail_cache_size

) The FE only keeps a limited number of query profiles in memory (default is typically 100). In high-concurrency environments, your query might be evicted from the cache before you can retrieve it. * Solution: Increase the cache size in the FE configuration:

sql
    ADMIN SET FRONTEND CONFIG ("query_detail_cache_size" = "500");

5. Reporting Delay Profile reporting from BE to FE is asynchronous. Even after a query finishes, there can be a sub-second delay before the BEs transmit the full profile to the FE. * Solution: If your automation is too fast, a small sleep (e.g., 500ms) before calling

SHOW PROFILELIST

or

get_query_profile()

often resolves the "missing" profile issue. Recommended Workflow To get the profile reliably via SQL (available in v3.0+):

sql
-- 1. Ensure variables are set
SET enable_profile = true;
SET is_report_success = true;

-- 2. Run your query
SELECT ... ;

-- 3. Retrieve the profile using the ID immediately
SELECT get_query_profile(last_query_id());

If you still can't see the profile, check the FE

sys_log/fe.warn.log

. If the BE fails to report the profile (e.g., due to network issues or RPC timeouts), it will often log a warning there. References • faq/Others.md • sql-reference/sql-statements/cluster-management/plan_profile/SHOW_PROFILELIST.md • administration/management/proc_profile.md • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_FRONTENDS.md • deployment/deploy_manually.md • administration/management/FE_configuration.md

@Rocky are there ways to find the leader without having an operator or node privilege role? SHOW FRONTENDS requires that

If you do not have SYSTEM-level OPERATE or cluster_admin privileges to run

SHOW FRONTENDS

or

SHOW PROC '/frontends'

, you can use the following methods to identify the Leader node or ensure you are getting the profile correctly: 1. Check

information_schema.fe_metrics

StarRocks exposes internal metrics via the

information_schema.fe_metrics

table (available in v3.1+). This table is often readable by users with basic metadata access. You can look for the metric that indicates leadership:

sql
SELECT * FROM information_schema.fe_metrics 
WHERE NAME LIKE '%is_leader%';

* Leader Node: The row where the

VALUE

is

1

belongs to the Leader FE. * Follower Node: The

VALUE

will be

0

. * Note: Since this table returns metrics for the node you are connected to, a

1

means your current connection is already on the Leader. 2. Use

get_query_profile()

(v3.0+) Instead of searching through the

SHOW PROFILELIST

or the Web UI (which are often local to the FE node you are on), use the SQL function. This is the most consistent way to retrieve a profile by ID regardless of node inconsistencies:

sql
-- 1. Run your query
SELECT ... ;

-- 2. Immediately retrieve the profile using the session's last query ID
SELECT get_query_profile(last_query_id());

_The

get_query_profile

function is designed to be the authoritative way to pull the profile data into your SQL client as a string._ 3. Check for specific session variables Sometimes a Follower FE has the query ID in its "list" but hasn't received the full report from the BEs. Ensure these are set to force reporting:

sql
SET enable_profile = true;
SET is_report_success = true; -- Ensures profiles are sent for successful queries
SET runtime_profile_report_interval = 1; -- Reports status every 1s (for long queries)

4. Metadata IP Variables In some environments, you can check system variables that might reveal the Leader/Master IP (though this depends on specific versioning and configurations):

sql
SHOW VARIABLES LIKE '%master%';
-- or
SHOW VARIABLES LIKE '%leader%';

Why "Consistent" connection to FE matters Even with a single connection, if your FE is a Follower, it only sees the queries that it personally handled. Profiles for queries handled by other FEs are sent to the Leader. * If you are connecting via a Load Balancer: Your "single connection" might be persistent, but if it landed on a Follower, you won't see the global query list. * If you are connecting to a fixed IP: Use the

fe_metrics

query above to confirm if that specific IP is the Leader. If it returns

0

, you should ask your admin for the IP of the node where

is_leader = 1

. Summary Recommendation If you can't get the Leader IP, stop using

SHOW PROFILELIST

and switch entirely to

SELECT get_query_profile(last_query_id())

. This bypasses the need to manually browse the profile list and is much more reliable for automation. References • sql-reference/information_schema/fe_metrics.md • deployment/deploy_manually.md • sql-reference/information_schema/be_metrics.md • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_FRONTENDS.md • sql-reference/System_variable.md