http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> can I use JWT token auth for HTTP s...
# questions-and-troubleshooting
r
@Rocky can I use JWT token auth for HTTP stream load endpoint
r
r
@Rocky I don't want to use user/password for HTTP SQL endpoint. I would like to use JWT token auth. How can I do this
r
To use JWT token authentication for the StarRocks HTTP SQL endpoint instead of a username and password, you must be using StarRocks v3.5.0 or later, which introduced support for JSON Web Token (JWT) authentication. Here is the step-by-step guide to setting this up: 1. Configure JWT in FE You need to configure the Frontend (FE) to trust your JWT issuer. You can do this by adding the following parameters to your 
fe/conf/fe.conf
file and restarting the FE: * 
jwt_jwks_url
: The URL to the JSON Web Key Set (JWKS) service or the local path to your public key file. * 
jwt_principal_field
: The field in the JWT claim that represents the StarRocks username (e.g., 
sub
or 
preferred_username
). * 
jwt_required_issuer
 (Optional): The expected issuer (
iss
) of the token. * 
jwt_required_audience
 (Optional): The expected audience (
aud
) of the token. Example `fe.conf`:
Copy code
properties
jwt_jwks_url = <http://your-auth-server/protocol/openid-connect/certs>
jwt_principal_field = sub
2. Create a User with JWT Authentication In StarRocks, you must create a user that is mapped to the identity provided in the JWT. The 
principal_field
value in your token must match the username you create.
Copy code
sql
-- Replace 'tom' with the value from the 'sub' field in your JWT
CREATE USER tom IDENTIFIED WITH authentication_jwt;
If you didn't set global configurations in 
fe.conf
, you can specify them per user:
Copy code
sql
CREATE USER tom IDENTIFIED WITH authentication_jwt AS
'{"jwks_url": "<http://your-auth-server/certs>", "principal_field": "sub"}';
3. Call the HTTP SQL API using the Bearer Token Once the user is created and JWT is configured, you can call the HTTP SQL API by passing the token in the 
Authorization
header as a 
Bearer
token. Endpoint: 
http://<fe_ip>:<fe_http_port>/api/v1/catalogs/default_catalog/databases/<db_name>/sql
Example using `curl`:
Copy code
bash
curl -X POST 'http://<fe_ip>:8030/api/v1/catalogs/default_catalog/databases/test_db/sql' \
   -H "Authorization: Bearer <your_jwt_token>" \
   -H "Content-Type: application/json" \
   -d '{"query": "SELECT * FROM my_table LIMIT 10;"}'
Important Notes: * Compatibility: This feature requires v3.5.0+. * Username Match: The value in the JWT's 
principal_field
must exactly match the StarRocks username. * No Password: Since the user is identified with 
authentication_jwt
, no password is required or used for this account when authenticating via token. References • developers/code-style-guides/restful-api-standard.md • sql-reference/http_sql_api.md • administration/user_privs/authentication/jwt_authentication.md • administration/user_privs/authentication/oauth2_authentication.md • administration/http_interface.md
✅ 1
r
@Rocky thank you - I'm using Pod identity and I'd like to create users with authentication_jwt relying on my cluster's OIDC provider
Copy code
CREATE USER `system:serviceaccount:my-namespace:my-service-account-name`
IDENTIFIED WITH authentication_jwt AS '{
  "jwks_url":         "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id/keys>",
  "principal_field":  "sub",
  "required_issuer":  "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id>",
  "required_audience": "<http://pods.eks.amazonaws.com|pods.eks.amazonaws.com>"
}';
I'm not able to create a user with this username however ( I believe the 
:
character is special ). What can I do instead?
r
Hello! I'm Rocky. The issue you're encountering is that StarRocks has strict naming conventions for usernames. According to the System Limits, usernames can only consist of digits (0-9), letters (a-z or A-Z), and underscores (_). The colon character 
:
is indeed a special character that is not currently permitted in StarRocks user identities. Since the 
sub
claim in Kubernetes OIDC tokens for service accounts follows the format 
system:serviceaccount:<namespace>:<name>
, it cannot be used directly as a username in StarRocks. Recommended Solution: Use a Different 
principal_field
 EKS OIDC tokens typically include additional claims that provide the service account name without the full 
system:serviceaccount
prefix. You can switch the 
principal_field
to use one of these claims instead. Common Kubernetes claims available in the token include: • `kubernetes.io/serviceaccount/name`: Contains only the name of the service account (e.g., 
my-service-account-name
). • `kubernetes.io/serviceaccount/namespace`: Contains only the namespace (e.g., 
my-namespace
). How to update your configuration: 1. Create the user using only the service account name (or an underscored version):
Copy code
sql
   CREATE USER `my_service_account_name` -- Use underscores or just the name
   IDENTIFIED WITH authentication_jwt AS '{
     "jwks_url":         "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id/keys>",
     "principal_field":  "<http://kubernetes.io/serviceaccount/name|kubernetes.io/serviceaccount/name>", -- Use the specific name claim
     "required_issuer":  "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id>",
     "required_audience": "<http://pods.eks.amazonaws.com|pods.eks.amazonaws.com>"
   }';
2. Note on Uniqueness: If you have multiple namespaces and need to ensure uniqueness, you'll need to be careful as StarRocks currently requires the 
principal_field
value to match the username exactly and doesn't support regex transformation or claim concatenation (like 
namespace_name
) within the JWT property itself. If this is a requirement, you might consider using a custom claim if you control the token generation, or using a unique naming convention for your service accounts across the cluster. Alternative: Security Integration (v3.4+) If you are on version 3.4 or later, you can also use Security Integration to manage JWT authentication globally, but the same naming restrictions apply to the "virtual users" or mapped users created during the login process. Let me know if you have more questions about how to map these claims! References • administration/management/FE_configuration.md • administration/user_privs/authentication/security_integration.md
r
@Rocky when I pass in a valid JWT token I get the following error: HTTP 401 Unauthorized 
{"status":"FAILED","code":"1","msg":"invalid Base64 input character: 46 (decimal)","message":"invalid Base64 input character: 46
. I'm making a request like:
Copy code
curl -X PUT <http://localhost:8030/api/mydatabase/mytable/_stream_load> -H "Content-Type: application/json" -H "Authorization: Bearer $(cat /path/tojwt/file)" -d '{}' -v
r
The error 
invalid Base64 input character: 46 (decimal)
occurs because the decimal value 46 corresponds to the dot character (
.
). A JWT token consists of three parts separated by dots (
header.payload.signature
). When you send the header 
-H "Authorization: Bearer <JWT>"
, the StarRocks Front-End (FE) is likely attempting to parse this header using its default Basic Authentication handler. Basic Auth expects a Base64-encoded string (which does not contain dots), and when the decoder encounters a dot in your JWT, it throws this error. Here are the reasons why this is happening and how to resolve it: 1. Version Requirement StarRocks introduced support for JWT authentication in v3.5.0. If you are using a version earlier than v3.5.0, JWT is not supported, and the FE will only recognize Basic Authentication (username and password). * Action: Check your StarRocks version using 
SELECT aurora_version();
or 
SHOW FRONTENDS;
. If it's earlier than 3.5.0, you must use Basic Auth. 2. Stream Load Authentication Support Even in v3.5.0+, JWT authentication is primarily supported for the MySQL protocol and specific HTTP interfaces (like the Web UI and JDBC over HTTP). The Stream Load API (
/api/.../_stream_load
) in many current implementations still expects Basic Authentication. The standard way to authenticate a Stream Load request is via the 
-u
flag in 
curl
, which generates a 
Basic
auth header:
Copy code
bash
curl --location-trusted -u "username:password" -X PUT ...
3. JWT Configuration (v3.5.0+) If you are on v3.5.0 or later and want to use JWT, ensure the following are correctly configured: * FE Configuration: Your 
fe.conf
must have JWT properties configured (e.g., 
jwt_jwks_url
). * User Creation: The user must be created with the JWT authentication method:
Copy code
sql
  CREATE USER 'your_user' IDENTIFIED WITH authentication_jwt;
* Principal Mapping: Ensure the 
sub
(subject) claim in your JWT matches the StarRocks username exactly. Recommendation If you are trying to automate loading and want to avoid using a plaintext password: 1. Use Basic Auth: This is the most reliable method for Stream Load. You can pass the credentials via 
curl -u "user:pass"
. 2. Check for "Bearer" support: If you are on 3.5.0+, verify if your specific build supports 
Bearer
tokens for the Stream Load endpoint. If the error persists, it indicates the endpoint is still defaulting to the Basic Auth parser. 3. Ensure no newlines: If you continue with the file approach, ensure the JWT file doesn't have hidden trailing newlines:
Copy code
bash
   # Use -n to avoid newlines if manually creating, or use $(cat ... | tr -d '\n')
   -H "Authorization: Bearer $(cat /path/tojwt/file | tr -d '\n')"
Summary: The FE is misinterpreting your 
Bearer
token as an invalid 
Basic
auth string. For Stream Load, stick to Basic Authentication (
-u username:password
) unless you are on a version that explicitly documents 
Bearer
support for the 
_stream_load
endpoint. References • sql-reference/sql-functions/crytographic-functions/to_base64.md • sql-reference/sql-functions/crytographic-functions/base64_decode_string.md • sql-reference/sql-functions/crytographic-functions/from_base64.md • sql-reference/sql-functions/crytographic-functions/base64_decode_binary.md • sql-reference/sql-functions/bitmap-functions/base64_to_bitmap.md • administration/Meta_recovery.md
r
@Rocky Can I use the JWT token as my password when using basic credentials?
Copy code
curl -X PUT <http://localhost:8030/api/mydatabase/mytable/_stream_load> -H "Content-Type: application/json" -u "your_user:$(cat /path/tojwt/file)" -d '{}' -v
When I do this, I get HTTP 401 Unauthorized, 
{"status":"FAILED","code":"1","msg":"Access
denied for your_user@127.0.0.1","message":"Access denied for
your_user@127.0.0.1"}
r
No, you should not use the JWT token as a password in the basic credentials (the 
-u
flag) for Stream Load. Using 
-u "your_user:token"
tells 
curl
to send a Basic Authentication header (
Authorization: Basic <base64>
). StarRocks then interprets this as a standard username and literal password login. When you provide a JWT as the password, StarRocks fails to authenticate it because it expects a standard password hash comparison. Starting from StarRocks v3.5.0, JWT authentication is supported for REST APIs (including Stream Load). To use it correctly, you should pass the token in the 
Authorization: Bearer
 header instead. Recommended Steps: 1. Verify Version: Ensure you are using StarRocks v3.5.0 or later. 2. Create the User: The user must be created with the 
authentication_jwt
method.
Copy code
sql
    CREATE USER your_user IDENTIFIED WITH authentication_jwt AS
    '{
      "jwks_url": "<http://your-auth-server/certs>",
      "principal_field": "sub"
    }';
_(Note: The 
principal_field
in your JWT, typically 
sub
, must match the StarRocks username 
your_user
.)_ 3. Correct 
curl
Syntax: Use the 
-H "Authorization: Bearer ..."
header and remove the 
-u
flag.
Copy code
bash
    curl -X PUT "<http://localhost:8030/api/mydatabase/mytable/_stream_load>" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $(cat /path/tojwt/file)" \
      -H "Expect: 100-continue" \
      -T /path/to/data.json -v
Why you received the error: * Version: If you are on a version earlier than v3.5.0, JWT authentication is not supported. * Auth Type Mismatch: Even in v3.5.0+, the system expects the token in a 
Bearer
header for REST APIs. When sent via 
-u
, the backend tries to validate it as a standard password, which results in 
Access denied
. * Principal Match: Ensure the 
sub
(or your configured 
principal_field
) inside the JWT matches the username exactly. References • developers/code-style-guides/restful-api-standard.md • administration/management/FE_configuration.md • administration/user_privs/authentication/oauth2_authentication.md • administration/management/BE_configuration.md • best_practices/authentication_authorization.md
r
@Rocky I've created a non-root user and want to use username:password to access stream load endpoint. I get 401 Unauthorized what could i do instead? Response: 
{"status":"FAILED","code":"1","msg":"Access denied for user \u0027tenant-user\u0027@\u0027%\u0027. Please ask the admin to grant permission(s) or try activating existing roles using \u003cset [default] role\u003e. Current role(s): []. Inactivated role(s): [root].","message":"Access denied for user \u0027tenant-user\u0027@\u0027%\u0027. Please ask the admin to grant permission(s) or try activating existing roles using \u003cset [default] role\u003e. Current role(s): []. Inactivated role(s): [root]."}
I did 
GRANT root to tenant-pipeline
r
Hi there! Rocky here to help you out. The error message you're seeing indicates that while the 
root
role has been granted to your user, it is currently inactivated by default. In StarRocks, simply granting a role doesn't always make it active automatically, especially for HTTP-based operations like Stream Load where you cannot manually run a 
SET ROLE
command before the request. To fix this, you need to set the 
root
role as a default role for that user. This ensures the role is automatically activated whenever the user connects or sends a request. Solution 1: Set the Default Role Run the following SQL command (as a user with administrative privileges):
Copy code
sql
-- Replace 'tenant-user'@'%' with the exact user identity from your error message
SET DEFAULT ROLE 'root' TO 'tenant-user'@'%';
Or using the 
ALTER USER
syntax:
Copy code
sql
ALTER USER 'tenant-user'@'%' DEFAULT ROLE 'root';
Solution 2: Activate all roles on login (Global) If you want all users to have their granted roles automatically activated upon login without having to set default roles individually, you can change this global setting:
Copy code
sql
SET GLOBAL activate_all_roles_on_login = TRUE;
Why this happens: * Inactivated Roles: StarRocks allows a user to have many roles, but only "Default Roles" are active at the start of a session. * Stream Load: Since Stream Load is a single HTTP request, it relies on the user's default configuration. If the required permissions (like 
INSERT
) are inside a role that isn't set as default, the request will fail with "Access denied." Note: Based on your error message, ensure you are granting the role to the correct user identity. Your error mentions 
tenant-user
while your command mentioned 
tenant-pipeline
. Make sure they match! References • faq/loading/Stream_load_faq.md • administration/user_privs/authorization/privilege_faq.md • administration/user_privs/authorization/user_privs.md • administration/user_privs/authorization/User_privilege.md • loading/StreamLoad.md • sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md
r
@Rocky i'm on 4.0.8-2090bfc. Using the HTTP sql endpoint with bearer token (JWT) fails. Event creating a user like so, using 
jti
as principal as it's at the top-level of the claim:
Copy code
CREATE USER `jti-uuid`
   IDENTIFIED WITH authentication_jwt AS '{
     "jwks_url":         "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id/keys>",
     "principal_field":  "jti", 
     "required_issuer":  "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id>",
     "required_audience": "starrocks"
   }';
I get the same illegal base64 decimal character error. When I parse out the second part of the token by doing: 
curl <http://mystarrockscluster:8030/api/catalogs/default_catalog/databases/yipyip/sql> -H "Content-Type: application/json" -H "Authorization: Bearer $(cat /var/run/secrets/tokens/starrocks/starrocks-token | cut -d. -f2)" -H "Content-Type: application/json" -d '{"query": "SELECT * FROM mytable LIMI
T 10;"}' -v
I get:
Copy code
:"Access denied for {\"aud\"@192.168.87.73","message":"Access denied for {\"aud\"@192.168.87.73"}
r
Hello! I'm Rocky, your StarRocks assistant. It looks like you're trying to use JWT authentication with the StarRocks HTTP SQL endpoint on version 4.0.8, but encountering issues with the 
Authorization: Bearer
header. Based on the errors you're seeing and the current implementation of the StarRocks HTTP SQL API, here is what’s happening and how to fix it: The Problem The 
Authorization: Bearer
scheme is not natively supported by the StarRocks HTTP SQL endpoint (
/api/v1/.../sql
) in a way that automatically extracts the principal from the JWT. Currently: 1. The "illegal base64" error: When you send a full JWT (
header.payload.signature
), the dots (
.
) in the token are causing the FE's header decoder to fail, as it expects a standard base64 string. 2. The "Access denied for {JSON}" error: When you manually extracted the payload (
cut -d. -f2
), StarRocks successfully decoded the base64 part but treated the entire resulting JSON string as the StarRocks username. It did not extract the 
jti
claim because it isn't treating that header as a JWT; it's treating it like an opaque identity string. The Solution: Use Basic Auth with the JWT as the Password To use JWT authentication with the HTTP SQL API, you should use the Basic Authentication format. In StarRocks, when a user is created with 
IDENTIFIED WITH authentication_jwt
, the FE expects the full JWT token to be passed as the password for that specific user. 1. Verify your User Setup Ensure the StarRocks username matches the value of the 
jti
claim in your token.
Copy code
sql
CREATE USER `jti-uuid`
IDENTIFIED WITH authentication_jwt AS '{
  "jwks_url": "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id/keys>",
  "principal_field": "jti", 
  "required_issuer": "<https://oidc.eks.us-east-2.amazonaws.com/id/cluster-id>",
  "required_audience": "starrocks"
}';
2. Correct 
curl
Command Pass the username and the full token (with all three parts) using the 
-u
flag. 
curl
will automatically format this into a 
Basic
auth header.
Copy code
bash
# Get the full token
TOKEN=$(cat /var/run/secrets/tokens/starrocks/starrocks-token)

# Use Basic Auth: -u 'username:password' where password is the full JWT
curl -X POST "<http://mystarrockscluster:8030/api/v1/catalogs/default_catalog/databases/yipyip/sql>" \
  -H "Content-Type: application/json" \
  -u "jti-uuid:$TOKEN" \
  -d '{"query": "SELECT * FROM mytable LIMIT 10;"}'
Key Requirements * Full Token: Do not 
cut
the token. Use the entire string: 
header.payload.signature
. * Username Mapping: The username provided in the 
-u
flag (
jti-uuid
) must exactly match the value inside the 
jti
claim of the JWT you are providing. * FE Config: If you are using a custom 
principal_field
globally, make sure 
jwt_principal_field
is set correctly in your 
fe.conf
, though the 
CREATE USER ... AS '{...}'
syntax you used overrides this for that specific user. If you continue to see the "illegal base64" error even when using 
-u
, ensure your token doesn't contain unusual trailing whitespace or hidden characters from the 
cat
command. Using 
echo -n
or ensuring the environment variable is clean can help. References • administration/user_privs/authentication/jwt_authentication.md • sql-reference/http_sql_api.md • administration/user_privs/authentication/security_integration.md
Open in Slack
PreviousNext
Powered by