Hi everyone, I'm encountering issues with configu...
# ask-ai
h
Hi everyone, I'm encountering issues with configuring Apache Pulsar to authenticate using Keycloak with OpenID Connect (OIDC). The Docker container for Pulsar is running well, but I'm facing problems with authentication. I have set up the following configuration for the Pulsar broker and the Spring Boot application, but I'm running into issues with authentication. Broker Configuration (broker.conf): # Specify the authentication provider authenticationProviders=org.apache.pulsar.broker.authentication.oidc.AuthenticationProviderOpenID # Enable authentication authenticationEnabled=true # Disable authorization (if not needed) authorizationEnabled=false # OpenID Connect configuration oidc.auth.provider.openid.issuerUrl=http://your-keycloak-server/realms/master oidc.auth.provider.openid.clientCredentialsFile=file:///path/to/your/client-credentials.json oidc.auth.provider.openid.audience=account # Allowed token issuers and audiences openIDAllowedTokenIssuers=http://your-keycloak-server/realms/master openIDAllowedAudiences=account # Optional settings openIDRoleClaim=sub openIDRequireIssuersUseHttps=false # Token validation settings authenticationRefreshCheckSeconds=120 openIDAcceptedTimeLeewaySeconds=120 openIDKeyIdCacheMissRefreshSeconds=120 Spring Boot Application Configuration: URL issuerUrl = new URL("http://your-keycloak-server/realms/master"); URL credentialsUrl = new URL("file:///path/to/your/client-credentials.json"); String audience = "account"; System.out.println("PulsarClient initializing..."); // Build Pulsar Client with OAuth2 Authentication return PulsarClient.builder() .serviceUrl("pulsar://your-pulsar-server:6650") .authentication(AuthenticationFactoryOAuth2.clientCredentials(issuerUrl, credentialsUrl, audience)) .operationTimeout(30, TimeUnit.SECONDS) .connectionTimeout(30, TimeUnit.SECONDS) .build(); Credentials JSON File (client-credentials.json): { "client_id": "client_id", "client_secret": "client_secret", "type": "client_credentials", "grant_type": "client_credentials", } Issues Encountered: Authentication Failures: Errors indicating Failed to authenticate the client and Failed to get partitioned topic metadata. JWT Claims Missing: Errors about missing nbf (not before) claim in the JWT. Configuration Problems: Errors such as Required configuration parameter: privateKey and UnsupportedAuthenticationException. Questions: Missing Claims: Why might the JWT tokens issued by Keycloak be missing the nbf claim, and how can this be resolved? Authentication Failures: What could be causing the Failed to authenticate the client errors, and how can I troubleshoot this? Private Key Error: How can I resolve the Required configuration parameter: privateKey issue when using AuthenticationProviderOpenID? Any insights or suggestions would be greatly appreciated. Thank you! (edited)
l
@RunLLM please reply to this question
r
To address your issues with configuring Apache Pulsar to authenticate using Keycloak with OpenID Connect (OIDC), let's break down the problems and potential solutions: 1. Missing Claims in JWT Tokens The
nbf
(not before) claim is optional in JWT tokens. If Keycloak is not including this claim, you can either configure Keycloak to include it or adjust Pulsar's settings to not require it. According to the OpenID Connect documentation, Pulsar validates several claims, but
nbf
is not mandatory. 2. Authentication Failures Authentication failures can be due to several reasons: - Incorrect Configuration: Ensure that the
issuerUrl
,
clientCredentialsFile
, and
audience
are correctly set in both the Pulsar broker and the Spring Boot application. - Token Validation: Verify that the tokens issued by Keycloak are valid and contain the necessary claims (
iss
,
aud
,
exp
, etc.). You can use tools like jwt.io to decode and inspect the tokens. 3. Private Key Error The error
Required configuration parameter: privateKey
suggests that Pulsar is expecting a private key for token validation. However, when using OIDC, the public key from the issuer's JWKS endpoint should be used instead. Ensure that the
oidc.auth.provider.openid.clientCredentialsFile
is correctly pointing to the JSON file with client credentials. Example Configuration Here is an example configuration for the Pulsar broker and Spring Boot application: Broker Configuration (broker.conf):
Copy code
# Specify the authentication provider
authenticationProviders=org.apache.pulsar.broker.authentication.oidc.AuthenticationProviderOpenID

# Enable authentication
authenticationEnabled=true

# Disable authorization (if not needed)
authorizationEnabled=false

# OpenID Connect configuration
oidc.auth.provider.openid.issuerUrl=http://your-keycloak-server/realms/master
oidc.auth.provider.openid.clientCredentialsFile=file:///path/to/your/client-credentials.json
oidc.auth.provider.openid.audience=account

# Allowed token issuers and audiences
openIDAllowedTokenIssuers=http://your-keycloak-server/realms/master
openIDAllowedAudiences=account

# Optional settings
openIDRoleClaim=sub
openIDRequireIssuersUseHttps=false

# Token validation settings
authenticationRefreshCheckSeconds=120
openIDAcceptedTimeLeewaySeconds=120
openIDKeyIdCacheMissRefreshSeconds=120
Spring Boot Application Configuration:
Copy code
URL issuerUrl = new URL("http://your-keycloak-server/realms/master");
URL credentialsUrl = new URL("file:///path/to/your/client-credentials.json");
String audience = "account";

System.out.println("PulsarClient initializing...");

// Build Pulsar Client with OAuth2 Authentication
return PulsarClient.builder()
        .serviceUrl("pulsar://your-pulsar-server:6650")
        .authentication(AuthenticationFactoryOAuth2.clientCredentials(issuerUrl, credentialsUrl, audience))
        .operationTimeout(30, TimeUnit.SECONDS)
        .connectionTimeout(30, TimeUnit.SECONDS)
        .build();
Credentials JSON File (client-credentials.json):
Copy code
{
  "client_id": "client_id",
  "client_secret": "client_secret",
  "type": "client_credentials",
  "grant_type": "client_credentials"
}
Would you like more detail on any specific part of the configuration or troubleshooting steps? Sources: - Authentication using OpenID Connect - GitHub Issue: Pulsar Authorization with Keycloak 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.