Hi there, I am trying to deploy the helm chart on an EKS cluster with all AWS managed services. For the datahub-gms deployment, you need access to the Glue schema registry. Unfortunately you need AWS credentials for this. I want to use least privileged credentials by associating an IAM role to a service account. I configured the IAM role and have this in my values.yml:

datahub-gms:
  enabled: true
  image:
    repository: ${local.ecr_image_prefix}/linkedin/datahub-gms
    tag: ${local.datahub_image_version}
  serviceAccount:
    create: true
    name: "datahub-gms"
    annotations:
      <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: ${module.iam_eks_role.iam_role_arn}

I tested that my service account works correctly by spinning up a test AWS CLI pod (last step in here: https://aws.amazon.com/premiumsupport/knowledge-center/eks-pods-iam-role-service-accounts/) However, my datahub-gms pod is still crash looping and is still assuming my worker role, instead of my least privileged IAM role:

Caused by: software.amazon.awssdk.services.glue.model.AccessDeniedException: User: arn:aws:sts::XXX:assumed-role/mixed-1-node-group-20220530082254040100000001/i-08d04210802a93209 is not authorized to perform: glue:GetSchemaByDefinition on resource: arn:aws:glue:eu-west-3:XXX:registry/neo-datahub-kafka-pro because no identity-based policy allows the glue:GetSchemaByDefinition action (Service: Glue, Status Code: 400, Request ID: XXX)

I am wondering if anyone has encountered this before and if maybe the application doesn’t use the default credentials provider chain?

Would this help? https://datahubproject.io/docs/deploy/aws#aws-glue-schema-registry AWS glues doesn’t support sigv4 unfortunately 😞

Didn’t know about this! Thanks a lot! 😄

Yeah. Because of the numerous limitations of glue schema registry (biggest one being that it does not support python sdk), we’ve been pushing people towards deploying the confluent schema registry (which is free)