Hi Team! I need help! 🙇‍♂️ Before I ask my questions, I want to let you know my current situation because of the significant time difference that may cause delays in our communication. I am trying to deploy Datahub on AWS following the deployment guide. I have launched three EKS nodes(v1.24.9-eks) using Terraform. When I ran

helm install prerequisites

, the ‘prerequisites-cp-schema-registry’ pod kept failing and restarting, while other pods remained in the pending state. It was exactly the same issue mentioned in this thread on Slack. Although I added the EBS usage policy to EKS, PVC binding did not work, and when I ran the

kubectl get pv

command, no PVs were found. Then I checked

kubectl get storageclasses

and found an StorageClass with the name ‘standard.’ I finally succeeded in binding the PVC only after modifying the values.yaml file as follows, and I could see that

prerequisites-cp-schema-registry

was running normally.

elasticsearch:
	...
# # Request smaller persistent volumes.
  volumeClaimTemplate:
   accessModes: ["ReadWriteOnce"]
   storageClassName: "standard"
   resources:
     requests:
       storage: 30Gi

...

mysql:
  enabled: true
  auth:
    # For better security, add mysql-secrets k8s secret with mysql-root-password, mysql-replication-password and mysql-password
    existingSecret: mysql-secrets
  global:
    storageClass: "standard"

I have deployed Datahub once on EKS version v1.23.13-eks with Datahub version 9.x using the same method. At that time, when I checked PVCs, the bound sc name was gp2 (default). However, when I added the EBS policy to EKS, PVC was bound immediately without modifying the values.yaml file. So,

my first question

is: With the EKS version upgrade, I couldn’t see gp2 as the default storage class, and there was only ‘standard’ (which doesn’t have a default name). Therefore, I added the storageClass option in the values.yaml file to solve the issue. However, I’m wondering if creating a separate default storage class with VOLUMEBINDINGMODE set to WaitForFirstConsumer is the correct solution. `Second question`: The Kubernetes deployment document recommends installing ‘prerequisites’ and then installing ‘datahub/datahub’ using helm, citing dependencies. However, when RDS(mysql), MSK, and OpenSearch are already set up, should I set ‘enabled’ to true or false for es and mysql in the prerequisites values.yaml file? `Third question`: Do EKS nodes need to be at least 3 in number? Also, is it necessary to have 3 or more Kafka brokers? Thank you for taking the time to read through my lengthy question. If there is any part of my inquiry that you didn’t fully understand, please feel free to ask for clarification. I look forward to your response.

This content can't be displayed.

Hi, you should point towards your existing deployments for the prereqs, and I believe the cluster numbers can be scaled down

@astonishing-answer-96712 Thanks I tried that solution and it automatically deployed pvc, pv.