Hello everyone, we’re trying to bring Datahub to p...
# troubleshoot
w
Hello everyone, we’re trying to bring Datahub to production and struggle with the Kubernetes setup. The issue is basically very simple - we can’t make any ingestion work. It does not execute ingestion tasks and Status is shown as “Pending...“. datahub-gms shown only one single log line: 123544.420 [I/O dispatcher 1] INFO c.l.m.s.e.update.BulkListener:47 - Successfully fed bulk request. Number of events: 4 Took time ms: -1 Trying the same on a quickstart installation works fine. Given our lack of familiarity with how Datahub work on the inside we’re stuck. Does that look like a common problem? Raising log verbosity did not provide us with additional insights. Any help is much appreciated. Thanks!
a
Hi, what version- is the quickstart working on K8's or just locally?https://datahubproject.io/docs/deploy/kubernetes/#quickstart
w
Hi @astonishing-answer-96712 - thank you for getting back to me. I simply ran the quickstart locally and it worked fine. As the second step I ran the Helm based deployment on my Docker for Desktop Kubernetes cluster and it also successfully ingested data from a DB. The next step will be to redeploy it to our EKS cluster and see if it properly ingests data there. It might be not before Tuesday until I get the chance to try that out. Will let you know as soon as I have some results.
Hi @astonishing-answer-96712 just a quick update - we managed to deploy Datahub and for now we’re just stuck with ArgoCD not allowing resource names to be longer than 52 characters, which is violated by “datahub-application-datahub-restore-indices-job-template”. Not sure how to get around this, but it is rather an ArgoCD issue than a Datahub issue. Thank you being on stand-by to help us out. Have a great week!
a
Hmm, what resources in particular are having trouble? Are the urn names just too long?
This may be a good bug to raise with the team, however it is an interesting Constraint on the ArgoCD side
w
The resource ist the job “datahub-application-datahub-restore-indices-job-template”, which as ArgoCD works is prefixed with the Argo Application Name (in our case “datahub-application”. That made the resource name longer than 52 characters and ArgoCD denied to deploy it
Indeed it would make sense IMHO to not let resource names grow too long