Hi team, I’ve tried following the documentation an...
# questions-and-troubleshooting
g
Hi team, I’ve tried following the documentation and attempted both OAuth 2.0 and JWT authentication using Keycloak, but neither approach is working. When I try to connect to StarRocks, I receive an “access denied” error, even though the user exists in Keycloak. Has anyone successfully implemented OAuth with StarRocks as described in the documentation? If so, could you please share your configuration details or any additional steps required? https://docs.starrocks.io/docs/administration/user_privs/authentication/security_integration/#connect-to-starrocks-via-a-security-integration . Thanks in advance
Hi team, I’ve tried following the documentation and attempted both OAuth 2.0 and JWT authentication using Keycloak, but neither approach is working. When I try to connect to StarRocks, I receive an “access denied” error, even though the user exists in Keycloak. Has anyone successfully implemented OAuth with StarRocks as described in the documentation? If so, could you please share your configuration details or any additional steps required? https://docs.starrocks.io/docs/administration/user_privs/authentication/security_integration/#connect-to-starrocks-via-a-security-integration . @Rocky
r
Hi! I'm Rocky, and I'd be happy to help you troubleshoot your Keycloak integration. The "access denied" error when using OAuth 2.0 or JWT with Keycloak is common and usually stems from a mismatch between the token claims and the StarRocks configuration. Here are the most critical points to check based on the documentation: 1. Verify the
principal_field
This is the most common cause of "access denied." StarRocks extracts the username from the token field specified by
principal_field
(default is
sub
). * The Issue: In Keycloak, the
sub
claim is typically a UUID (e.g.,
262572ca-....
), but you likely created the user in StarRocks with a human-readable name like
tom
. * The Fix: Most Keycloak setups use
preferred_username
for the human-readable login name. Ensure your security integration or user creation uses:
Copy code
sql
    "principal_field" = "preferred_username"
Also, ensure the value in this field exactly matches the username you created in StarRocks. 2. Check
required_issuer
and
required_audience
StarRocks is very strict about these claims if they are provided: * Issuer (
iss
):
This must match the
iss
claim in your Keycloak token exactly. Be careful with trailing slashes (e.g.,
<http://keycloak:8080/realms/master>
vs `http://keycloak:8080/realms/master/`). * Audience (
aud
):
This usually matches your Keycloak
client_id
. If the token's
aud
claim does not match the
required_audience
property in StarRocks, access will be denied. 3. Client Version and Connection Method (JWT) If you are using JWT authentication via a MySQL client: * MySQL Client Requirement: You must use MySQL Client version 9.2 or later. * Connection Command: You must use the specific OIDC plugin flag to pass the token:
Copy code
bash
    mysql -h <fe_ip> -P 9030 \
      --authentication-openid-connect-client-id-token-file=/path/to/token.txt \
      -u <username>
4. User Creation Syntax Ensure the user in StarRocks was created specifically to use the security integration or the OAuth/JWT method. * Using Security Integration:
Copy code
sql
    CREATE USER 'tom' IDENTIFIED WITH 'my_keycloak_integration';
* Manual JWT/OAuth2:
Copy code
sql
    CREATE USER 'tom' IDENTIFIED WITH authentication_jwt AS '{...}';
5. Troubleshooting Steps To narrow down the root cause, I recommend: 1. Decode your JWT: Go to jwt.io and paste the token you are using. Check the
iss
,
aud
, and
preferred_username
(or
sub
) fields. 2. Check FE Logs: Look at the
fe.log
on your StarRocks leader FE. It often provides a more descriptive reason for the "access denied" error (e.g., "issuer mismatch" or "user not found"). 3. Validate JWKS URL: Ensure the StarRocks FE can actually reach the
jwks_url
provided in your configuration (e.g., via
curl
from the FE machine). If you can share the
CREATE SECURITY INTEGRATION
statement you used (redacting the secret), I can help you spot any syntax or configuration issues! Referencesadministration/user_privs/authentication/jwt_authentication.mdadministration/user_privs/authentication/oauth2_authentication.md
g
CREATE SECURITY INTEGRATION keycloak_oauth -> PROPERTIES ( -> "type" = "authentication_oauth2", -> "auth_server_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/auth", -> "token_server_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/token", -> "client_id" = "starrocks-client", -> "client_secret" = "***", -> "redirect_url" = "https://sdpplynn01.techsophy.com:8443/api/oauth2", -> "jwks_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/certs", -> "principal_field" = "preferred_username", -> "required_issuer" = "https://auth-uni.techsophy.com/auth/realms/datalake", -> "required_audience" = "starrocks-client", -> "comment" = "Keycloak OAuth2 integration" -> );
@Rocky i shared the security integration please check
r
It looks like you haven't shared the configuration snippet yet! Please paste the
CREATE SECURITY INTEGRATION
statement or the property settings you'd like me to review. In the meantime, here are a few key things I can check for you once you provide the details: *1. Supported Types* StarRocks currently supports three main types of security integrations: * LDAP (
type
=
authentication_ldap_simple
) * JWT (
type
=
authentication_jwt
) * OAuth 2.0 (
type
=
authentication_oauth2
) *2. Common Checklist* * Authentication Chain: After creating the integration, did you update the global variable
authentication_chain
to include your new integration? * Example:
SET GLOBAL authentication_chain = (native, <your_integration_name>);
* External Authorization: Note that when using Security Integration, external authorization (like Apache Ranger) is currently required. Integration with StarRocks' native RBAC is not yet supported for these virtual users. * Property Accuracy: * For LDAP: Ensure
authentication_ldap_simple_bind_base_dn
and
authentication_ldap_simple_user_search_attr
are correctly mapped to your directory schema. * For JWT/OAuth 2.0: Verify the
jwks_url
is accessible by the FE and the
principal_field
matches the StarRocks username. Please share your SQL or configuration, and I'll be happy to look for any syntax errors or missing required properties! Referencesadministration/user_privs/authentication/security_integration.md
g
@Rocky here is the snippet CREATE SECURITY INTEGRATION keycloak_oauth -> PROPERTIES ( -> "type" = "authentication_oauth2", -> "auth_server_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/auth", -> "token_server_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/token", -> "client_id" = "starrocks-client", -> "client_secret" = "*", -> "redirect_url" = "https://sdpplynn01.techsophy.com:8443/api/oauth2", -> "jwks_url" = "https://auth-uni.techsophy.com/auth/realms/datalake/protocol/openid-connect/certs", -> "principal_field" = "preferred_username", -> "required_issuer" = "https://auth-uni.techsophy.com/auth/realms/datalake", -> "required_audience" = "starrocks-client", -> "comment" = "Keycloak OAuth2 integration" -> );
r
The snippet you provided looks like a valid SQL statement to create a Security Integration for Keycloak OAuth2 in StarRocks. This approach is the recommended way to manage OAuth2 configurations centrally rather than defining them for each individual user. Key Points for your Configuration: 1.
type
: Correctly set to
"authentication_oauth2"
. 2.
redirect_url
: Ensure that
<https://sdpplynn01.techsophy.com:8443>
is the correct address and port (usually the FE HTTP/HTTPS port) of your StarRocks Leader FE. The path
/api/oauth2
is mandatory. 3.
principal_field
: You've set this to
"preferred_username"
. This means the value in the
preferred_username
claim of the Keycloak JWT must exactly match the username created in StarRocks. 4.
client_secret
: Ensure you replace
"*"
with the actual secret from your Keycloak client configuration. How to use it: Once the security integration is created, you can create a user that uses this integration for authentication:
Copy code
sql
-- Create a user and associate it with the security integration
CREATE USER 'your_keycloak_username' IDENTIFIED WITH authentication_oauth2 AS 'keycloak_oauth';
Tips for Keycloak: * Issuer URL: The
required_issuer
must match the
iss
claim in the JWT exactly (including trailing slashes if present in Keycloak). * Audience: The
required_audience
must match the
aud
claim in the JWT. * JWKS: Ensure the StarRocks FE can reach the
jwks_url
to verify the token signatures. Does this configuration result in any specific error, or are you looking to verify if the setup is correct? Referencesadministration/user_privs/authentication/security_integration.mdadministration/user_privs/authentication/oauth2_authentication.md
g
Copy code
sql
-- Create a user and associate it with the security integration
CREATE USER 'your_keycloak_username' IDENTIFIED WITH authentication_oauth2 AS 'keycloak_oauth';
why i need to run this command , Is the user suppose to be taken from keyclock right. as per the documentation no need to create users in starrocks @Rocky
r
Hi there! I'm Rocky, and I can help clarify how user management works with Keycloak and OAuth 2.0 in StarRocks. The confusion arises because StarRocks (starting from v3.5.0) supports two different ways to integrate with external identity providers like Keycloak. Depending on which approach you choose, you either do or do not need to manually create users. 1. Manual User Mapping (The command you're using) The command
CREATE USER 'username' IDENTIFIED WITH authentication_oauth2 AS 'keycloak_oauth';
is used when you want a persistent local user in StarRocks that delegates its password check to Keycloak. * Why use this? It’s useful if you have a small number of users and you want to manage their permissions (using
GRANT
) directly inside StarRocks. * Process: You must create the user first. When the user logs in, StarRocks checks with Keycloak to verify their identity. 2. Security Integration (The "No Manual Creation" way) If your goal is to have users automatically "taken from Keycloak" without running
CREATE USER
for every person, you should use the Security Integration feature. In this mode, StarRocks creates a virtual session-scoped user the moment someone successfully authenticates via Keycloak. To set this up, you follow these steps: 1. Create a Security Integration: Instead of creating individual users, you create one global integration:
Copy code
sql
    CREATE SECURITY INTEGRATION keycloak_integration
    PROPERTIES (
        "type" = "authentication_oauth2",
        "auth_server_url" = "https://<keycloak-host>/auth/realms/<realm>/protocol/openid-connect/auth",
        "token_server_url" = "https://<keycloak-host>/auth/realms/<realm>/protocol/openid-connect/token",
        "client_id" = "starrocks-client",
        "client_secret" = "your-secret",
        "jwks_url" = "https://<keycloak-host>/auth/realms/<realm>/protocol/openid-connect/certs",
        "principal_field" = "preferred_username" -- This identifies the user name
    );
2. Enable the Authentication Chain: You must tell StarRocks to use this integration for logins that don't match a local user:
Copy code
sql
    ADMIN SET FRONTEND CONFIG ("authentication_chain" = "keycloak_integration");
Which one should you use? * Use the
CREATE USER
command
if you want to explicitly control who can log in and assign them static permissions within StarRocks. * Use
SECURITY INTEGRATION
if you have a large number of users and you want to manage access via User Groups or Apache Ranger. Note on Authorization: If you use the Security Integration (no manual
CREATE USER
), the "virtual" user will have no permissions by default. You will typically need to configure a Group Provider so that StarRocks can map the user's Keycloak groups to StarRocks Roles automatically. Feel free to ask if you'd like more details on setting up the Group Provider! Referencesadministration/user_privs/authentication/security_integration.mdadministration/management/FE_configuration.mdbest_practices/authentication_authorization.md
g
If you use the Security Integration (no manual
CREATE USER
), the "virtual" user will have no permissions by default. You will typically need to configure a Group Provider so that StarRocks can map the user's Keycloak groups to StarRocks Roles automatically. @Rocky how can i configure the group provider to starrocks roles
r
To configure a Group Provider in StarRocks to map external groups (like those from Keycloak) to StarRocks roles, you need to follow a three-step process: create the group provider, link it to your security integration, and then grant roles to the external groups. Starting from StarRocks v3.5.0, the Group Provider feature decouples group management from authentication, allowing you to fetch group information from systems like LDAP or a local file even when using SSO (OAuth2/JWT). 1. Create a Group Provider A Group Provider tells StarRocks where to find the mapping between users and groups. If your Keycloak is backed by LDAP, an LDAP Group Provider is the best choice. If not, you can use a File Group Provider. Option A: LDAP Group Provider (Recommended if Keycloak uses LDAP)
Copy code
sql
CREATE GROUP PROVIDER keycloak_ldap_provider 
PROPERTIES (
    "type" = "ldap",
    "ldap_conn_url" = "ldap://<ldap_host>:<port>",
    "ldap_bind_root_dn" = "cn=admin,dc=example,dc=com",
    "ldap_bind_root_pwd" = "password",
    "ldap_bind_base_dn" = "dc=example,dc=com",
    "ldap_group_filter" = "(&(objectClass=groupOfNames)(cn=starrocks_*))",
    "ldap_group_identifier_attr" = "cn",
    "ldap_group_member_attr" = "member",
    "ldap_user_search_attr" = "uid=([^,]+)" -- Extracts username to match OAuth2 principal
);
Option B: File Group Provider (For manual mapping) If you don't have LDAP, you can define a mapping file (e.g.,
groups.txt
in
fe/conf/
): File content (
fe/conf/groups.txt
):
Copy code
text
analytics_team:user1,user2,user3
admin_team:admin_user
SQL command:
Copy code
sql
CREATE GROUP PROVIDER keycloak_file_provider 
PROPERTIES (
    "type" = "file",
    "group_file_url" = "groups.txt"
);
2. Link to your Security Integration You must associate the group provider with your existing Keycloak Security Integration so that StarRocks knows to look up groups when a user logs in via SSO.
Copy code
sql
ALTER SECURITY INTEGRATION keycloak_integration SET (
    "group_provider" = "keycloak_ldap_provider" -- or "keycloak_file_provider"
);
3. Map Groups to StarRocks Roles Once the connection is established, use the
GRANT
statement to map the external group name to a StarRocks role. Users belonging to that group will automatically inherit these permissions upon login.
Copy code
sql
-- Map the external group "analytics_team" to the StarRocks role "db_admin"
GRANT db_admin TO EXTERNAL GROUP analytics_team;

-- Map the external group "finance" to the StarRocks role "read_only"
GRANT read_only TO EXTERNAL GROUP finance;
Key Considerations * Principal Matching: Ensure that the
principal_field
defined in your Security Integration (e.g.,
preferred_username
or
email
) matches the user identifier returned by your Group Provider. If the OAuth2 principal is
jdoe
but the LDAP provider returns
John Doe
, the mapping will fail. * Virtual Users: Because these users are "virtual" (no manual
CREATE USER
), they exist only during their session. Their permissions are derived entirely from the
EXTERNAL GROUP
grants. * Version Note: The Group Provider feature was introduced in v3.5.0. Ensure your StarRocks cluster is on a compatible version. Referencesadministration/user_privs/group_provider.mdbest_practices/authentication_authorization.md
1