@Rocky when it set up the flink to pipe data to starrocks, but it hit redirect when it try to set up stream with FE node in the eks. So I have to make CN resolvable and make flink talk to CN directly. What is recommended way?

When using the StarRocks Flink connector on EKS, the

307 Temporary Redirect

occurs because the FE (Front End) assigns a CN/BE (Compute/Backend Node) to handle the data stream. By default, the FE returns the internal pod IP or internal DNS name of that CN, which your Flink cluster (if external or in a different network) cannot resolve. There are three recommended ways to handle this, with the first two being the most common for Kubernetes environments: 1. Enable

proxy_load

(Easiest / Configuration-only) Starting from StarRocks 2.5, the FE can act as a proxy for Stream Load requests. Instead of redirecting Flink to the CN's internal address, the FE will receive the data and forward it to the CN itself. In your Flink SQL DDL or DataStream options, add the following property:

sql
-- Flink SQL example
CREATE TABLE starrocks_sink (
    ...
) WITH (
    'connector' = 'starrocks',
    'load-url' = 'fe_lb_address:8030', -- Use your FE LoadBalancer or Service address
    'sink.properties.proxy_load' = 'true', -- This header forces FE to proxy the data
    ...
);

Why use this: Flink only needs to be able to reach the FE. It never needs to resolve or talk to the CNs directly. 2. Deploy the FE Proxy (Recommended for K8s) If you are using the StarRocks Operator or Helm chart, you can deploy the FE Proxy component. This is an Nginx-based proxy designed specifically to handle external loading tools. It provides a single stable endpoint and correctly maps the internal redirection. In your Helm `values.yaml`:

yaml
starrocks:
  starrocksFeProxySpec:
    enabled: true
    service:
      type: LoadBalancer # Or NodePort

Then, point your Flink connector's

load-url

to the FE Proxy Service address (usually port 8080). 3. Configure FQDNs If you prefer Flink to talk to CNs directly (which can be more performant as it avoids the FE bottleneck), you must ensure the CNs are registered with resolvable Fully Qualified Domain Names (FQDNs) rather than IPs. * Configure StarRocks to use FQDNs for node registration (e.g., setting

broker_client_use_fqdn=true

in FE/BE configs, though this applies more to brokers, the general practice is ensuring

show backends

shows hostnames). * Ensure your Flink nodes use a DNS resolver that can resolve the EKS internal service names (e.g.,

starrocks-cn-0.starrocks-cn-search.svc.cluster.local

). Summary Recommendation * For the fastest fix: Use

'sink.properties.proxy_load' = 'true'

. * For a production-grade K8s setup: Enable the

starrocksFeProxySpec

via the Operator. It is specifically built to solve the "external client vs. internal K8s redirect" problem. References • loading/Flink-connector-starrocks.md • loading/Kafka-connector-starrocks.md • quick_start/helm.md

@Rocky how it works under the hood, the flink expect to set up a stream, and will FE ping to one CN or rotate to different CN?

Under the hood, StarRocks uses a redirection-based coordination model for Stream Load (which the Flink connector uses). Here is exactly how it works: 1. The Initial Handshake (FE Polling) When the Flink connector starts a load (typically at each checkpoint or when a batch is ready), it sends an HTTP

PUT

request to the FE (Front End). * The FE's Role: It acts as a scheduler. It does not handle the data itself. * Node Selection: The FE uses a polling (round-robin) mechanism to select an available CN (Compute Node) or BE (Backend) to act as the "Coordinator" for that specific load request. * The Rotation: The FE does rotate among all healthy CNs to ensure no single node is overwhelmed with coordinating all the data traffic. 2. The 307 Redirect Instead of taking the data, the FE responds to Flink with an HTTP 307 Temporary Redirect. * This redirect message contains the address of the selected CN (e.g., `http://10.0.x.x:8040/...`). * The "EKS Problem": In Kubernetes, the FE knows the CNs by their internal Pod IPs. It sends that internal IP back to Flink. If Flink is outside the VPC or cannot resolve those Pod IPs, the redirect fails. 3. Data Transfer (The Stream) Once the Flink connector receives the 307 redirect, it automatically follows it: * Flink opens a new connection directly to the CN specified in the redirect. * The data "stream" happens here. Flink pushes the raw data to the CN. * The CN receives the data, parses it, and distributes it to the other nodes where the actual data buckets reside. 4. Summary of the Lifecycle 1. Flink → FE: "I have data to load." 2. FE: (Selects CN-A via round-robin) "Go talk to CN-A at

10.0.1.5

." 3. Flink → CN-A: "Here is the data." 4. CN-A: (Commits data) "Success." 5. (Next batch) Flink → FE: "I have more data." 6. FE: (Selects CN-B via round-robin) "Go talk to CN-B at

10.0.1.6

." Why

proxy_load

or

FE Proxy

is recommended: When you use

sink.properties.proxy_load = 'true'

, you change this behavior: * Flink → FE: "I have data to load (and please proxy it)." * FE: (Selects CN-A) FE keeps the connection open with Flink and opens a separate connection to CN-A. * Data flow: Flink → FE → CN-A. * Benefit: Flink only ever talks to the FE's LoadBalancer IP. The internal CN IPs are never exposed to Flink, avoiding the resolution issue entirely. References • loading/StreamLoad.md • quick_start/shared-data.md • quick_start/routine-load.md • administration/management/resource_management/Load_balance.md • loading/loading_introduction/loading_concepts.md