http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> if the principal field value from a...
# questions-and-troubleshooting
b
@Rocky if the principal field value from a jwt authentication aren't valid starrocks user names what are my options?
r
If the principal field values in your JWT (e.g., the 
sub
claim) do not match valid StarRocks usernames or contain characters that StarRocks rejects (such as colons 
:
, which are known to be problematic for AWS IAM ARNs in 
sub
claims), you have several options to resolve this. In StarRocks (v3.5.0 and later), the 
principal_field
value is mapped directly to the StarRocks username. Currently, there is no built-in regex transformation or mapping logic within the FE configuration. Option 1: Map to a Different JWT Claim If your JWT contains other claims that follow a valid username format (e.g., 
preferred_username
, 
nickname
, or 
email
), you can configure StarRocks to use that field instead of the default 
sub
. * In `fe.conf`:
Copy code
properties
    jwt_principal_field = preferred_username
* In a 
CREATE USER
statement:
Copy code
sql
    CREATE USER tom IDENTIFIED WITH authentication_jwt AS
    '{
      "jwks_url": "...",
      "principal_field": "preferred_username"
    }';
Option 2: Configure your Identity Provider (IdP) Most IdPs (Keycloak, Okta, Auth0, etc.) allow you to add custom claims to the ID Token. You can configure your IdP to generate a claim specifically for StarRocks (e.g., 
starrocks_username
) that strips out invalid characters or matches your pre-created StarRocks users. Option 3: Use Security Integration and Virtual Users If you use a Security Integration (available starting in v3.5.0), StarRocks treats the identity as a "virtual user" for that session. * Advantage: You don't necessarily have to pre-create every user in StarRocks. * Limitation: When using Security Integration for JWT, you currently must use an external authorization system like Apache Ranger to manage permissions. StarRocks' native RBAC does not yet support managing permissions for these dynamically mapped virtual users. Option 4: Create Users with Special Characters If the "invalid" characters are simply 
@
or 
.
, StarRocks does support these in usernames, but you must quote them correctly in your DDL. * Example: 
CREATE USER '<mailto:user.name@domain.com|user.name@domain.com>' IDENTIFIED WITH authentication_jwt;
* However, if the principal contains characters like 
:
or starts with digits in a way that violates SQL identifier rules, this will not work. Summary of Recommendations * If you have many users: Use a Security Integration combined with Apache Ranger. This avoids the need to manually map and create every user, delegating the complex identity logic to Ranger. * If you have a small number of users: Map the 
principal_field
to a cleaner claim (like 
email
) and create matching users in StarRocks manually using 
CREATE USER '<mailto:user@email.com|user@email.com>' IDENTIFIED WITH authentication_jwt
. authentication_jwt;`. * If the claims are totally unusable: Modify your IdP's token issuance policy to include a sanitized username claim that is specifically sanitized for StarRocks. References • administration/user_privs/authentication/ldap_authentication.md • administration/management/FE_configuration.md • administration/user_privs/authorization/ranger_plugin.md • release_notes/release-3.1.md • administration/user_privs/authentication/native_authentication.md
b
@Rocky I get this message: [42000][1064] Getting analyzing error. Detail message: invalid user name: person@place.com. Is that saying that the IdP saying it is an invalid user name or StarRocks
r
The error message you're seeing—`[42000][1064] Getting analyzing error. Detail message: invalid user name: person@place.com`—is definitely coming from StarRocks (the FE/Frontend node), not your Identity Provider (IdP). Here is the breakdown of why this is happening and how to fix it: 1. Why it's a StarRocks Error The code 
1064
and the prefix 
Getting analyzing error
are internal StarRocks error markers. They indicate that the SQL Analyzer (the part of the StarRocks engine that parses and validates queries/connections) has rejected the input. In this case, the FE has successfully decoded your JWT, looked at the field you designated as the 
principal_field
(or the default 
sub
), and found the string: 
<mailto:person@place.com|person@place.com>
. 2. Why StarRocks Rejected It StarRocks has strict naming conventions for usernames (as defined in the System Limits documentation): * Allowed characters: Digits (
0-9
), letters (
a-z
, 
A-Z
), and underscores (
_
). * Characters in your principal: Your IdP is providing a Slack-style linkified string containing 
<
, 
>
, 
:
, 
|
, and 
@
. * Result: StarRocks sees these special characters and immediately flags the username as "invalid" because it doesn't conform to its internal identifier rules. 3. How to Fix It You have a few paths to resolve this, depending on your control over the IdP and your version of StarRocks: A. Change the 
principal_field
to a "Clean" Claim Check if your JWT contains a different field that has a simple username (e.g., 
preferred_username
, 
nickname
, or 
upn
). If your JWT has a field like 
"nickname": "person"
, update your configuration: * In fe.conf: 
jwt_principal_field = nickname
* In SQL: 
CREATE USER person IDENTIFIED WITH authentication_jwt ...
B. Configure your IdP (The "Cleanest" Fix) Your IdP is likely sending the email wrapped in a mailto URI or Slack-specific markdown. You should configure your IdP (Okta, Keycloak, Auth0, etc.) to: 1. Add a custom claim (e.g., 
starrocks_user
). 2. Use a "Token Transformation" or "Script" in the IdP to strip out the 
<mailto:...>
parts so that only 
person_place
or 
person
is sent. C. Create Users with Backticks (If only 
@
or 
.
were the problem) If the principal was just 
<mailto:person@place.com|person@place.com>
, you could try creating the user in StarRocks using backticks:
Copy code
sql
CREATE USER `<mailto:person@place.com|person@place.com>` IDENTIFIED WITH authentication_jwt;
Note: Even with backticks, StarRocks will still reject characters like 
<
, 
>
, and 
|
during the analyzer phase because they are reserved for other SQL operations. D. Use Security Integration (v3.5.0+) If you are using Security Integration with JWT, StarRocks allows more flexible mapping, but the internal "virtual user" it creates still needs to have a name that passes validation. If your principal is inherently "dirty" (contains 
<
, 
|
, etc.), you will almost always need to sanitize it at the IdP level before it reaches StarRocks. Summary The string 
<mailto:person@place.com|person@place.com>
is being treated literally as a username by StarRocks. Because it contains characters like 
:
and 
|
, the StarRocks analyzer throws an error. You must ensure the JWT claim being used as the 
principal_field
contains only alphanumeric characters and underscores. References • administration/management/FE_configuration.md • administration/user_privs/authentication/jwt_authentication.md
b
@Rocky I did use backticks and it still failed with this message. I am using starrocks v 4.0.9
r
In StarRocks (including version 4.0.9), backticks only allow you to use reserved keywords (like 
SELECT
or 
TABLE
) as identifiers. They do not permit the use of characters that are fundamentally illegal in the StarRocks metadata system, such as 
<
, 
>
, 
|
, and 
:
. The error occurs because StarRocks has a strict internal regular expression for usernames (alphanumeric and underscores). When it decodes your JWT and finds a string like 
<mailto:person@place.com|person@place.com>
, it rejects it before even attempting to match it to a user. Since you cannot use that specific string as a username, here are your only viable options: Option 1: Use a Different JWT Claim (Recommended) Most JWTs contain multiple fields. Instead of using the default 
sub
field (which in your case seems to contain Slack-style linkified metadata), check if your token has a 
preferred_username
, 
email
, or 
nickname
field. 1. Check your token: Use a tool like jwt.io to see the payload. 2. Update 
fe.conf
or the 
CREATE USER
statement:
Copy code
sql
    -- If your JWT has a field "nickname": "person"
    CREATE USER person IDENTIFIED WITH authentication_jwt AS
    '{
      "jwks_url": "...",
      "principal_field": "nickname"
    }';
Option 2: Sanitize at the Identity Provider (IdP) If you have access to your IdP (e.g., Okta, Keycloak, Auth0, AWS Cognito), you should configure a Token Transformation or a Custom Claim. * Create a new claim called 
starrocks_user
. * Write a small script in your IdP to strip the 
<mailto:...|...>
formatting and just return 
person_place_com
. * Set 
principal_field = starrocks_user
in StarRocks. Option 3: Security Integration + Apache Ranger If you are on v4.0.9, you can use Security Integration to handle JWT. * How it works: StarRocks treats the identity as a "virtual user." * The Catch: This requires Apache Ranger for authorization. Native StarRocks RBAC (the 
GRANT
system) cannot manage users with these special characters. * If you use Ranger, the "name" passed to Ranger will still be that messy string, but Ranger policies can be written to handle it. However, this is significantly more complex to set up. Option 4: Use the Email Only (If IdP allows) If you can get the IdP to send just 
<mailto:person@place.com|person@place.com>
without the 
<mailto:...|...>
wrappers, then backticks will work. * Supported: ``CREATE USER 
<mailto:person@place.com|person@place.com>
IDENTIFIED WITH authentication_jwt;`` * NOT Supported: ``CREATE USER 
<mailto:person@place.com|...>
...`` (The 
|
and 
<
characters are hard-limit rejections). Summary for v4.0.9: StarRocks' internal engine cannot store a username containing 
|
or 
<
. You must change the 
principal_field
to point to a claim that contains a "clean" name or modify your IdP to provide one. References • administration/user_privs/authentication/jwt_authentication.md • administration/management/FE_configuration.md • administration/user_privs/authentication/security_integration.md • administration/user_privs/authentication/oauth2_authentication.md
b
@Rocky is there a way to have users automatically created in starrocks when using an identity provider?
r
Yes, you can have users "automatically" recognized without manually running 
CREATE USER
for every single person, but there is a major trade-off regarding how you manage their permissions. In StarRocks (v3.5.0 and later), this is achieved using Security Integration. 1. The "Automatic" Way: Security Integration Instead of creating individual users, you create a Security Integration object. When a user connects with a valid JWT or OAuth2 token, StarRocks validates the token and creates a virtual user for that specific session. * How it works: StarRocks looks at the 
principal_field
in the token and maps it to a session-based identity. * The Catch: Because these users don't "exist" permanently in StarRocks' internal metadata, the native RBAC system (
GRANT
/ 
REVOKE
statements) cannot see them to assign permissions. * The Requirement: To use this "automatic" approach, you must use Apache Ranger for authorization. Ranger allows you to define policies for users and groups that StarRocks will check at runtime, even if the user wasn't pre-created in the database. Example Syntax:
Copy code
sql
CREATE SECURITY INTEGRATION my_jwt_integration
PROPERTIES (
    "type" = "authentication_jwt",
    "jwks_url" = "<https://example.com/.well-known/jwks.json>",
    "principal_field" = "preferred_username"
);
2. The "Manual" Way: Pre-creating Users If you want to use StarRocks' native permission system (standard SQL 
GRANT
statements), you must manually create each user. * Can you automate this? There is no built-in "JIT Provisioning" that automatically writes a new row to the internal user table upon first login. * Common Workaround: Most organizations automate this via their CI/CD pipeline or a synchronization script (e.g., a Python script that pulls users from an IdP and runs 
CREATE USER IF NOT EXISTS ... IDENTIFIED WITH authentication_jwt
in StarRocks). 3. Handling Your Username Format Issue Even with "automatic" virtual users via Security Integration, the 
principal_field
value still needs to be usable by StarRocks to identify the user session. * If your IdP is sending 
<mailto:user@place.com|user@place.com>
, StarRocks will still attempt to use that as the session username. * If you are using Apache Ranger, Ranger can often handle complex strings, but StarRocks itself may still struggle to process the session if the username contains characters that break its internal session handling. Recommendation If you have a large, dynamic user base and don't want to manage 
CREATE USER
statements: 1. Set up Apache Ranger for StarRocks. 2. Use Security Integration to allow session-based "virtual users." 3. Configure your IdP to send a "clean" username (like just 
user_name
) in a custom claim so that the virtual user name is simple and readable. References • administration/user_privs/authentication/security_integration.md • administration/user_privs/authorization/ranger_plugin.md • administration/user_privs/authentication/jwt_authentication.md • administration/user_privs/group_provider.md References • using_starrocks/jit.md • administration/user_privs/authentication/native_authentication.md • administration/user_privs/authentication/jwt_authentication.md
b
@Rocky if i'm using ranger with virtual users would i still be able to pass credentials to a catalog, like apache polaris
r
Yes, you can. In fact, using Security Integration with Apache Polaris (or any Iceberg REST Catalog) is the intended architectural pattern for "zero-trust" or identity-propagated access in StarRocks. When you use "Virtual Users" (users authenticated via JWT who aren't manually created in StarRocks) and Apache Ranger, here is how the credential flow works: 1. The Identity Flow * Authentication: The user logs into StarRocks using their JWT token. Because you have a Security Integration defined, StarRocks validates the token but doesn't require a local 
CREATE USER
entry. * Authorization (Ranger): StarRocks sends the user's principal (from the JWT) to Apache Ranger. Ranger checks if that user has the 
USAGE
permission on the Catalog object. * Credential Pass-through: If authorized, StarRocks takes the original JWT token used by the user to log in and "passes it through" to Apache Polaris. 2. Required Catalog Configuration To make this work, you must tell the StarRocks Catalog to propagate the JWT. In your Polaris/Iceberg REST Catalog properties, include these specific settings:
Copy code
sql
CREATE EXTERNAL CATALOG polaris_catalog
PROPERTIES (
    "type" = "iceberg",
    "iceberg.catalog.type" = "rest",
    "iceberg.catalog.uri" = "<https://your-polaris-endpoint/api/catalog>",
    -- This is the key: tell StarRocks to pass the user's JWT to Polaris
    "iceberg.catalog.security" = "jwt",
    -- This allows Polaris to give StarRocks temporary S3/GCS/Azure credentials
    "iceberg.catalog.vended-credentials-enabled" = "true",
    -- Tell StarRocks to let Polaris handle the internal table/view permissions
    "catalog.access.control" = "allowall"
);
3. How Polaris Sees the "Virtual User" When StarRocks passes the JWT to Polaris: 1. Polaris receives the same token the user used to log into StarRocks. 2. Polaris decodes it and sees the same identity. 3. Polaris applies its own policies. If the user is 
<mailto:person@place.com|person@place.com>
in the JWT, Polaris will check if 
<mailto:person@place.com|person@place.com>
has access to the specific Iceberg namespace or table. 4. Role of Apache Ranger In this setup, Ranger acts as the "Gatekeeper" for the StarRocks side: * Ranger's Job: Verify the user is allowed to even see the catalog named 
polaris_catalog
. * Polaris's Job: Verify the user is allowed to read the data inside the tables. Summary of Benefits * No StarRocks User Management: You don't need to run 
CREATE USER
for 1,000 employees. * No Static Secrets: You don't store AWS/S3 keys in StarRocks. Polaris "vends" (issues) temporary storage credentials to the StarRocks BE nodes only for the duration of the query. * Single Source of Truth: If you revoke a user's access in your IdP, their JWT becomes invalid, and they instantly lose access to both StarRocks and the data in Polaris. One Caveat: Ensure your Polaris server is configured to trust the same JWKS (Public Keys) or Issuer as your StarRocks Security Integration so it can successfully validate the passed-through token. References • data_source/catalog/iceberg/iceberg_rest_security.md • data_source/catalog/unified_catalog.md
b
@Rocky can keycloak take the place of apache ranger?
r
No, Keycloak cannot take the place of Apache Ranger for authorization in StarRocks. They perform two completely different roles in a security stack: * Keycloak (Authentication): It is an Identity Provider (IdP). Its job is to verify who the user is and provide a token (JWT) to StarRocks. * Apache Ranger (Authorization): It is an Access Control engine. Its job is to decide what that user can do (e.g., "Can this user run 
SELECT
on 
db1.table1
?"). In StarRocks v4.0.9, the relationship between these and "Virtual Users" (automatic creation) is strict: 1. The "Virtual User" Requirement If you want users to be automatically recognized in StarRocks without running 
CREATE USER
for every person: * You must use a Security Integration (JWT or OAuth2). * When using Security Integration, StarRocks requires Apache Ranger for authorization. * Why? Because StarRocks' native permission system (
GRANT/REVOKE
) needs a persistent user record in its own metadata to attach permissions to. Since "virtual users" don't exist in the metadata, the internal system has nowhere to store their rights. Ranger stores these policies externally, allowing it to authorize users on the fly. 2. Can you use Keycloak without Ranger? Yes, but only if you pre-create the users (the "Manual" way): 1. Authentication: Keycloak sends the JWT to StarRocks. 2. Mapping: You must run 
CREATE USER 'user_name' IDENTIFIED WITH authentication_jwt
. 3. Authorization: You use StarRocks' native RBAC (
GRANT role TO user_name
). 4. Groups: You can use a Group Provider (v3.5+) to sync Keycloak/LDAP groups into StarRocks and use 
GRANT role TO EXTERNAL GROUP 'analytics_team'
. This way, you don't have to grant permissions to every individual, but the 
CREATE USER
step is still required for the system to recognize the login. 3. How they work together (The Enterprise Setup) If you want a fully automated system, the architecture looks like this: 1. Keycloak: Authenticates the user and issues a JWT containing their 
username
and 
groups
. 2. StarRocks Security Integration: Accepts the JWT and creates a "Virtual User" session. 3. StarRocks Group Provider: Pulls the user's groups from your directory (LDAP/File/Unix). 4. Apache Ranger: Receives the 
username
and 
groups
from StarRocks and checks its policies to see if that user (or any of their groups) has permission to the table. Summary Feature | With Keycloak Only | With Keycloak + Ranger Automatic User Creation | No (Virtual users not supported) | Yes (via Security Integration) Identity Verification | Yes (OIDC/SAML/JWT) | Yes Role/Permission Mgmt | Manual (
GRANT
in StarRocks) | Centralized (Ranger UI) Scalability | Hard (Must script 
CREATE USER
) | Easy (Purely policy-based) Verdict: If your goal is to avoid manual user creation entirely, you must use Apache Ranger alongside Keycloak. If you cannot use Ranger, you must pre-create users in StarRocks and point them to Keycloak for their password/JWT for their password-passwordless login. References • administration/user_privs/authentication/security_integration.md • administration/user_privs/authorization/ranger_plugin.md • administration/user_privs/group_provider.md • best_practices/authentication_authorization.md References • administration/user_privs/authorization/ranger_plugin.md
b
@Rocky is there an alternative to apache ranger that starrocks can use
r
As of the current version of StarRocks, Apache Ranger is the only supported third-party external authorization engine that integrates directly with the StarRocks privilege system. If you are looking for an alternative to avoid the complexity of Ranger, you have three primary architectural options depending on your goals: 1. The Native Alternative: RBAC + External Groups Instead of a separate authorization server like Ranger, you can use StarRocks’ Native RBAC combined with a Group Provider. This is the most popular alternative because it keeps authorization "inside" the database while still being automated. * How it works: You create a Security Integration (JWT/OAuth) for authentication and a Group Provider (LDAP/File/Unix) to pull user groups. * The Workflow: You don't grant permissions to individuals. Instead, you create a StarRocks Role (e.g., 
analyst_role
) and map it to an external group:
Copy code
sql
    GRANT 'analyst_role' TO EXTERNAL GROUP 'analytics_team';
* Trade-off: Unlike the "Virtual User" model with Ranger, you must still run a 
CREATE USER
(or have a script automate it) for the identity to be recognized by the native system. 2. The Catalog-Level Alternative: Iceberg REST / Polaris If your primary concern is authorizing access to external data (like Iceberg tables in S3), you can shift the "Alternative" to the Catalog level. * How it works: Use an Iceberg REST Catalog (like Apache Polaris). * The Workflow: StarRocks acts as a "dumb" passthrough for the user's JWT. Polaris (the catalog) then decides if the user has access to the data and "vends" temporary S3 credentials back to StarRocks. * Benefit: You manage permissions in the Catalog (Polaris) rather than in StarRocks or Ranger. 3. The Future Alternative: Custom Static Extensions (v4.1+) Starting in v4.1, StarRocks introduced Static Extensions. This is a developer-focused feature that allows you to write custom Java modules that plug into the StarRocks FE (Front End). * While there isn't a pre-built "OPA" (Open Policy Agent) plugin yet, the extension framework provides the hook points for a team with Java expertise to build a custom authorization bridge to other systems. Summary Comparison Goal | Solution | Complexity Fully Automated / Zero User Creation | Apache Ranger | High Centralized Groups / Internal RBAC | Group Provider | Low/Medium Data-Centric Authorization | REST Catalog (Polaris) | Medium Custom Homegrown Solution | Static Extensions (v4.1+) | High (Coding required) Note: If your requirement is strictly Virtual Users (users that exist only as a session and never in the database metadata), Apache Ranger is currently a hard requirement. Native RBAC cannot support virtualize.** References • administration/user_privs/authorization/ranger_plugin.md • administration/management/extensions.md
Open in Slack
PreviousNext
Powered by