Hi I got this error when try unload data from starrocks to g StarRocks #questions-and-troubleshooting

Hi I got this error when try unload data from star...

Phước Hồ

10/20/2025, 10:56 AM

Hi I got this error when try unload data from starrocks to gcs from cluster outside of GCP with GOOGLE_APPLICATION_CREDENTIALS environment variable config. Look like issue come from gcs-hadoop-connector lib

Copy code

INSERT INTO
FILES(
    "path" = "<gs://starrocks-storage/retention_3d/test/>",
    "format" = "parquet",
    "single" = "true",
    "fs.s3a.endpoint" = "<https://storage.googleapis.com>",
 "gcp.gcs.use_compute_engine_service_account" = "true"
)
SELECT * FROM tracking limit 10;

Copy code

IOException: Error reading credentials from stream, 'type' value 'external_account' not recognized. Expecting 'authorized_user' or 'service_account'.java.io.IOException: Error reading credential file from enviro
nment variable GOOGLE_APPLICATION_CREDENTIALS, value '/var/run/secrets/cloudsdk-config/cred.json': Error reading credentials from stream, 'type' value 'external_account' not recognized. Expecting 'authorized_user' or 'service_account'.

StarRocks version: 3.3.18

Phước Hồ

10/21/2025, 6:36 AM

hi, I think issue come from gcs-connector version

hadoop3-2.2.26

I tried patch image with version 3.0.13 and it work. Note: we using WIF as GOOGLE_APPLICATION_CREDENTIALS file and option "gcp.gcs.use_compute_engine_service_account" is set to false.

Phước Hồ

10/21/2025, 6:36 AM

Copy code

FROM maven:3.9.11-eclipse-temurin-11 AS build
WORKDIR /app
COPY test/pom.xml .

RUN mkdir -p "/opt/starrocks/be/lib/hadoop/common/"

RUN git clone <https://github.com/StarRocks/starrocks.git> && \
    cd starrocks && \
    git switch -c 3.3.18 3.3.18 && \
    cd java-extensions && \
    sed -i 's/<gcs.connector.version>hadoop3-2.2.26<\/gcs.connector.version>/<gcs.connector.version>3.0.13<\/gcs.connector.version>/' pom.xml && \
    mvn clean && \
    mvn --quiet package -DskipTests -T 16

RUN cp -r /app/starrocks/java-extensions/hadoop-lib/target/hadoop-lib/* /opt/starrocks/be/lib/hadoop/common/

FROM starrocks/be-ubuntu:3.3.18

RUN rm -rf /opt/starrocks/be/lib/hadoop/common/*.jar

COPY --from=build /opt/starrocks/be/lib/hadoop/common/ /opt/starrocks/be/lib/hadoop/common/

Phước Hồ

10/24/2025, 1:50 AM

I have added a PR for this. https://starrocks.slack.com/archives/C02FAD0JSSD/p1761062403283929

Satya Kuppam

11/03/2025, 5:04 PM

Hey I saw your PR for WIF and looks like it is still in flight. Did you figure out an alternate way to use WIF?

Phước Hồ

11/04/2025, 5:23 AM

Hi, I make it work by patch official image of starrocks like Dockerfile above I think it not recommend but it currently work for my use case. I copy the logic from build script of starrocks

Open in Slack

Previous Next