@Rocky is there a way to implement availability-zone awareness for STREAM LOADING? Meaning that data will be sent to nodes in specific / same availability zone to avoid cross AZ traffic.

As of the current version of StarRocks, there is no built-in, fully automated "availability-zone awareness" specifically for the Stream Load redirection logic inside the FE. However, you can achieve your goal of minimizing cross-AZ traffic by combining several StarRocks features and deployment strategies. 1. Control Client-to-BE Traffic (The Ingestion Point) When you submit a Stream Load request to a StarRocks FE, the FE uses a simple polling mechanism (round-robin) to redirect the client to a BE/CN that will act as the "Coordinator." This polling is not aware of the client's or the BE's availability zone. Solution: To ensure the data is first received by a node in the same AZ: * Direct Ingestion: Instead of sending the

curl

request to the FE, send it directly to the IP/Port of a BE or CN located in the same availability zone as your data source. * External Load Balancer: Use a cloud load balancer (like an AWS NLB or ALB) with zonal affinity (cross-zone load balancing disabled) in front of your BEs. Point your ingestion client to the LB, which will ensure the request stays within the same AZ. 2. Control Data Replica Distribution (The Storage Layer) Even if the data reaches a BE in the same zone, StarRocks must replicate that data to all tablet replicas (typically 3) for durability. If your replicas are spread across zones (which is recommended for HA), cross-AZ traffic is unavoidable during the write phase in a Shared-Nothing architecture. Solution: BE Labels (

labels.location

) You can use labels to manage where data lives: 1. Label your BEs:

sql
    ALTER SYSTEM MODIFY BACKEND "172.xx.xx.46:9050" SET ("labels.location" = "rack:az1");

2. Define Table Locality: When creating a table, specify that replicas should be placed in specific zones:

sql
    PROPERTIES ("labels.location" = "rack:az1,rack:az2,rack:az3");

This ensures that at least one replica is "local" to the ingestion AZ, but StarRocks will still perform cross-AZ writes to the other two replicas to ensure the transaction is committed safely across the cluster. 3. Shared-Data (Cloud-Native) Mode If you are using StarRocks in Shared-Data mode (separating storage and compute), the architecture changes: * Compute Nodes (CNs) process the data and write it to object storage (like Amazon S3 or Google Cloud Storage). * In this mode, there are no local "replicas" to sync between BEs. * Cross-AZ Optimization: If your CNs are in the same AZ as your data source, the ingestion traffic is local. The subsequent write from the CN to S3 is handled by the cloud provider's internal network (which often minimizes costs if using a VPC endpoint). Summary of Recommendations * To avoid Client -> StarRocks cross-AZ traffic: Send Stream Load requests directly to a BE in the same AZ or use a zone-aware Load Balancer. * To avoid StarRocks -> StarRocks cross-AZ traffic: This is not fully possible if you require cross-AZ High Availability (3 replicas in 3 AZs). However, using Shared-Data mode is the most effective way to reduce inter-node replication traffic, as data is written once to object storage rather than replicated between nodes. between BEs. For more details on node labeling, see the Add labels on BEs documentation. References • administration/management/resource_management/be_label.md • quick_start/helm.md • loading/StreamLoad.md • administration/management/FE_configuration.md

@Rocky

Direct Ingestion: Instead of sending the

curl

request to the FE, send it directly to the IP/Port of a BE or CN located in the same availability zone as your data source.

Will that work? Documentation specifies sending requests to fe_host:fe_port

Yes, it definitely works. While the standard practice is to send requests to the FE for automatic load balancing, the Direct Ingestion to a BE (Backend) or CN (Compute Node) is a fully supported feature. Here is how it works and what you need to change: 1. Change the Endpoint Instead of the FE IP and port (default

8030

), you use the IP of the specific BE/CN and its HTTP port (default

8040

). * FE URL:

http://<fe_ip>:8030/api/<db>/<table>/_stream_load

* BE/CN URL:

http://<be_ip>:8040/api/<db>/<table>/_stream_load

2. Why it helps with AZ Awareness When you send a request to the FE: 1. The FE receives the HTTP request. 2. The FE chooses a BE/CN via round-robin. 3. The FE sends an HTTP 307 Redirect back to your client. 4. Your client (if it follows redirects, like

curl -L

) then sends the data to the chosen BE. If the FE redirects your client in AZ-1 to a BE in AZ-2, you incur cross-AZ traffic for the entire data payload. By hitting the BE in the same AZ directly: * You skip the FE redirect entirely. * The data payload stays within the same AZ for the initial upload. 3. Important Considerations * Health Checks: If you point your client directly to a single BE IP and that BE goes down, your ingestion will fail. You should use a load balancer or a list of BE IPs within that specific AZ to maintain high availability. * Authentication: You still use the same credentials (

-u "user:passwd"

) as you would when hitting the FE. * Data Distribution: Even if the "Coordinator BE" (the one you hit) is in the same AZ, it will still distribute data to other BEs that hold the relevant tablet replicas. If those replicas are in different AZs, some cross-AZ traffic will still occur during the write phase (unless using Shared-Data mode as mentioned previously). The documentation explicitly notes this capability:

"You can also directly submit a load request on your client to a BE or CN of your choice."

For the exact port configurations, you can check your cluster's

be_http_port

(default

8040

) in the

be.conf

file or by running

SHOW BACKENDS;

. References • loading/StreamLoad.md • sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md • administration/management/BE_configuration.md References • quick_start/helm.md • administration/management/BE_configuration.md • sql-reference/sql-statements/loading_unloading/STREAM_LOAD.md • loading/StreamLoad.md