@Rocky query timeout not being applied to external catalogs. Is there something that I need to look into? I've given timeout of 60seconds and internal tables queries getting timedout by 60secs but external table hive metastore not getting timeout in 3mins as well and I set metastore_timeout of 30secs from hive. Is there something I'm missing here?

While the

query_timeout

session variable is the primary control for query duration in StarRocks, external catalogs (like Hive) involve several layers—metadata retrieval from the Metastore, file list fetching, and the actual data scan—which can sometimes bypass or delay the enforcement of this timeout if specific sub-timeouts are not configured. Based on the symptoms you've described, here is what you should check: 1. Planning vs. Execution Timeout If the query is taking 3+ minutes, it might be hanging during the Planning phase (metadata fetching) rather than the Execution phase (scanning).

query_timeout

typically governs the execution phase on the BEs. * Hive Metastore Timeout: StarRocks uses the FE configuration

hive_meta_store_timeout_s

(default is 10s) to control the connection timeout to the Hive Metastore. If you have millions of partitions or the HMS is slow, the FE might be blocked here. _ _Check:* Run

SHOW FRONTEND CONFIG LIKE 'hive_meta_store_timeout_s';

* JDBC Catalog Specifics: If you are using a JDBC-based catalog for metadata (though unlikely for Hive), check

jdbc_query_timeout_ms

and

jdbc_network_timeout_ms

(introduced in v3.3+). 2. External Scanner Library Timeouts StarRocks often uses JNI or HDFS/S3 clients to read external data. These clients have their own internal timeouts that might exceed 60 seconds. * HDFS Client Config: If your Hive data is on HDFS, the HDFS client configurations (like

ipc.client.connect.timeout

or

dfs.client.socket-timeout

) in your

hdfs-site.xml

can cause the process to wait significantly longer than the StarRocks query timeout. * Object Storage Request Timeout: If using S3/OSS, check the BE configuration: * `object_storage_request_timeout_ms`: The timeout for HTTP requests to object storage. * `object_storage_connect_timeout_ms`: The timeout for establishing socket connections. 3. Catalog Properties When you created the catalog, ensure you didn't set a property that overrides or confuses the timeout behavior. For a Hive catalog, StarRocks passes certain properties to the underlying Hadoop configuration. You mentioned setting

metastore_timeout

from Hive; ensure it is specified correctly in the catalog properties:

sql
ALTER CATALOG <catalog_name> SET PROPERTIES (
    "hive.metastore.client.socket.timeout" = "30" -- Unit is usually seconds
);

4. Known Limitation: Metadata Collection For some external catalogs (like Iceberg), there is a specific variable: * `metadata_collect_query_timeout`: Controls the timeout for collecting metadata. If a similar issue exists for Hive in your version, this is worth checking (though primarily for Iceberg/Delta). 5. Troubleshooting Steps 1. Identify where it hangs: Run the query and check the FE log (

fe.log

) and BE log (

<http://be.INFO|be.INFO>

). * If the FE log shows the query is still in the "Pending" or "Planning" state for 3 minutes, the issue is metadata retrieval (HMS connection or partition fetching). * If the query is "Running" on BEs, the issue is the scanner not checking the query deadline. 2. Verify the Session Variable: Double-check that

query_timeout

is actually set for that specific session by running

SELECT @@query_timeout;

just before your query. 3. Hedged Reads: If using HDFS, "Hedged Reads" can sometimes cause unexpected delays. You can try disabling them or tuning

hdfs_client_hedged_read_threadpool_size

on the BE side. If you are on an older version of StarRocks (e.g., pre-2.5), the external table implementation was less mature regarding timeout propagation. Upgrading to a 3.x version is highly recommended if you are seeing inconsistent timeout enforcement in catalogs. References • administration/management/FE_configuration.md • sql-reference/System_variable.md

@Rocky alter catalog unified_catalog_hms set properties ("hive.metastore.client.socket.timeout" = "30"); ERROR 1064 (HY000): Getting syntax error at line 1, column 38. Detail message: Unexpected input 'properties', the most similar input is {'('}. mysql>

The syntax error you encountered is because the

ALTER CATALOG

statement in StarRocks does not use the

PROPERTIES

keyword. The correct syntax uses

SET

followed directly by the properties in parentheses. 1. Correct the Syntax Error To update your catalog property, use this command:

sql
ALTER CATALOG unified_catalog_hms SET ("hive.metastore.client.socket.timeout" = "30");

(Note: Ensure the value is a string, e.g.,

"30"

. Depending on your StarRocks version, the unit for this specific Hive property is typically seconds.) ────────── 2. Why the timeout might be ignored If your internal queries timeout at 60s but Hive queries are hanging for 3+ minutes, it is likely because the query is stuck in the Planning phase (Metadata retrieval) rather than the Execution phase.

query_timeout

primarily governs the execution of the scan on the Backends (BEs). Here are the specific configurations you should check: A. FE Metadata Timeout (Planning Phase) When you query an external Hive table, the Frontend (FE) must connect to the Hive Metastore (HMS) to fetch partition information and file locations. *

hive_meta_store_timeout_s

: This is a global FE configuration that controls the connection timeout to the HMS. The default is

10s

. _ _Check it with:*

ADMIN SHOW FRONTEND CONFIG LIKE 'hive_meta_store_timeout_s';

_ _Note:* This is a static config in

fe.conf

. If it's set very high or not respecting the limit, the FE might block indefinitely. B. HDFS/Object Storage Client Timeouts If the FE successfully plans the query but the BE "hangs" during the scan, the underlying HDFS or S3 client might be waiting for a response and ignoring the StarRocks

query_timeout

. * HDFS Socket Timeout: If your data is on HDFS, the BE uses a Hadoop client. You may need to set

dfs.client.socket-timeout

(default 60s) or

ipc.client.connect.timeout

in your

hdfs-site.xml

or via Catalog properties. * BE Configs: Check

object_storage_request_timeout_ms

(for S3/OSS) if your Hive catalog points to object storage. C. Metadata Collection Timeout (v3.3+) In newer versions of StarRocks, there is a session variable specifically for metadata collection: *

metadata_collect_query_timeout

: Though primarily documented for Iceberg, check if setting this helps your Hive session:

sql
    SET metadata_collect_query_timeout = 30;

Summary of Troubleshooting 1. Check where it's hanging: Run

SHOW PROC '/current_queries';

while the query is running. If the

State

remains

Pending

or

Planning

for a long time, the issue is the FE communicating with the Metastore. 2. Verify Catalog Properties: After running the

ALTER CATALOG

command above, verify it with

SHOW CREATE CATALOG unified_catalog_hms;

. 3. Check FE Logs: Look at

fe.log

. If you see "get table metadata timeout" or similar entries, the FE is indeed hitting a wall with HMS. References • data_source/catalog/hive_catalog.md • administration/management/FE_configuration.md • sql-reference/sql-statements/Catalog/SHOW_CREATE_CATALOG.md References • administration/Meta_recovery.md • administration/management/FE_configuration.md • faq/shared_data_faq.md