@Rocky Is

object_storage_request_timeout_ms

parameter related to DB Backup?

Yes, the

object_storage_request_timeout_ms

parameter is directly related to database backup and restore operations when you are using an object storage repository (such as AWS S3, Google Cloud Storage, or Alibaba Cloud OSS). While it is sometimes categorized under "Query Engine" in documentation because it affects data lake queries, it serves a critical role in the underlying communication for BACKUP and RESTORE jobs: 1. What it controls This Backend (BE) configuration parameter defines the timeout duration for HTTP connections between the StarRocks BE nodes and the object storage service. * Default:

-1

(uses the default timeout of the underlying SDK, usually around 5–10 seconds). * Scope: It affects individual HTTP requests made by the BE to upload or download data files during a backup/restore job. 2. Relation to Backup Errors If your backup job fails with an error like

curlCode: 28, Timeout was reached

or

Fail to copy... msg: Timeout was reached

, it is often because individual file operations are taking longer than the allowed HTTP timeout. This can happen when: * The backup involves very large files (tablets). * The network connection to your object storage is slow or unstable. * The object storage service is experiencing high latency. 3. How it differs from other timeouts It is important to distinguish this from the FE-level job timeout: *

backup_job_default_timeout_ms

(FE): This is the total time allowed for the entire backup job to complete. *

object_storage_request_timeout_ms

(BE): This is the timeout for a single HTTP request (e.g., uploading one file part). If this times out, the BE may retry, but frequent timeouts will eventually cause the entire job to fail. Recommendation If you encounter timeout errors during a backup to S3-compatible storage, you can increase this value in your

be.conf

and restart the BE nodes. A common troubleshooting value is

30000

(30 seconds) or higher:

conf
object_storage_request_timeout_ms = 30000

_For more details on backup configuration, you can refer to the StarRocks documentation on Back up and Restore._ References • administration/management/FE_configuration.md • administration/management/BE_configuration.md • faq/operation_maintenance_faq.md • unloading/Export.md • sql-reference/sql-statements/loading_unloading/BROKER_LOAD.md

@Rocky How about

make_snapshot_worker_count

?

Yes,

make_snapshot_worker_count

is also directly related to database backups. While

object_storage_request_timeout_ms

handles the network communication with your storage,

make_snapshot_worker_count

controls the local concurrency on the Backend (BE) during the initial phase of a backup. Role in the Backup Process When you run a

BACKUP

command, the process happens in stages. This parameter affects the very first stage: 1. Snapshot Creation (Local): Before any data is uploaded to S3 or HDFS, each BE must create a consistent "snapshot" (hard links of the data files) of the tablets it hosts. 2.

make_snapshot_worker_count

determines how many threads on each BE can perform this local snapshotting task simultaneously. 3. Upload: Once snapshots are ready, they are uploaded to the repository (controlled by

upload_worker_count

). Key Details * Default Value:

5

* Where to configure:

be.conf

* Impact: If you have a very large number of tablets on a single BE, the default value of 5 might be a bottleneck, causing the "Snapshot" phase of the backup job to take a long time. * Optimization: You can increase this value (e.g., to 16 or 32) to speed up the snapshotting phase, provided the BE has enough CPU and I/O resources. Summary of Related BE Parameters If you are tuning backup performance, these three parameters work together: *

make_snapshot_worker_count

: How many local snapshots can be prepared at once. *

upload_worker_count

: How many files can be uploaded to the remote repository at once. *

object_storage_request_timeout_ms

: How long to wait for each individual upload request before timing out. _For more details, see the StarRocks BE Configuration documentation._ References • administration/management/BE_configuration.md • administration/management/Backup_and_restore.md

@Rocky how about max_upload_task_per_be?

Yes,

max_upload_task_per_be

is also directly related to database backups, but it functions at a different level than the parameters we discussed previously. While

make_snapshot_worker_count

and

upload_worker_count

are Backend (BE) settings that control local thread counts,

max_upload_task_per_be

is a Frontend (FE) configuration parameter. How it works This parameter acts as a "governor" or "throttle" from the FE's perspective when it is orchestrating a backup job: * Role: It defines the maximum number of concurrent upload tasks that the FE will assign to any single BE node for a single BACKUP operation. * Default:

0

(which means no limit is imposed; the FE will assign as many tasks as needed). * Introduced in: v3.1.0. Why use it? You would typically use this parameter to prevent a single backup job from overwhelming your BE nodes or your network. * If you have a very large cluster and a massive table, the FE might try to trigger hundreds of upload tasks on a single BE simultaneously. * By setting

max_upload_task_per_be

(e.g., to

10

), you ensure the FE only gives each BE 10 tasks at a time. As one task finishes, the FE sends the next one. Summary: The "Backup Control Chain" To give you the full picture, here is how these parameters interact during a backup: 1.

max_upload_task_per_be

(FE): Limits how many total tasks the leader FE pushes to a BE node. 2.

make_snapshot_worker_count

(BE): Limits how many of those tasks can prepare their local data files (snapshots) at the same time. 3.

upload_worker_count

(BE): Limits how many of those tasks can actually stream data to S3/HDFS at the same time. 4.

object_storage_request_timeout_ms

(BE): Ensures that if a single upload request takes too long, it times out rather than hanging indefinitely. Note: There is a corresponding parameter for restores called

max_download_task_per_be

, which performs the same throttling function during the

RESTORE

process. References • administration/management/FE_configuration.md • administration/management/BE_configuration.md