Hello! I'm trying to do a batch ingestion from S3, our cluster lives in AWS EKS, encountered this issue when running

pinot-admin.sh LaunchDataIngestionJob -jobSpecFile=./job-spec.yml

:

software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=mothership-dev, profileFile=ProfileFile(profilesAndSectionsMap=[{default=Profile(name=default, properties=[s3, region, aws_access_key_id, aws_secret_access_key]), mothership-dev=Profile(name=mothership-dev, properties=[sso_region, aws_access_key_id, sso_registration_scopes, aws_secret_access_key, region, sso_start_url, sso_role_name, sso_account_id])}, {}])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=mothership-dev, profileFile=ProfileFile(profilesAndSectionsMap=[{default=Profile(name=default, properties=[s3, region, aws_access_key_id, aws_secret_access_key]), mothership-dev=Profile(name=mothership-dev, properties=[sso_region, aws_access_key_id, sso_registration_scopes, aws_secret_access_key, region, sso_start_url, sso_role_name, sso_account_id])}, {}])): To use Sso related properties in the 'mothership-dev' profile, the 'sso' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]

any one has any clues? we are using a service account for the pinot cluster that can access s3

Hey @Chris Hao can you guide me how to integrate SSO in pinot