https://datahubproject.io logo
Join Slack
Powered by
Hello, currently I’m installing datahub in gcp and...
# troubleshoot
a
Hello, currently I’m installing datahub in gcp and using helm chart but in my workloads I got these errors, did anyone know how to fix this?
👀 1
✅ 1
a
Hi, what versions are you using and have you followed this guide? https://datahubproject.io/docs/deploy/gcp
a
Yes I followed that guide and I’m using 1.23.14-gke.1800 stable channel
Update: Already found the problem before because when I ran command 
kubectl create secret generic mysql-secrets --from-literal=mysql-root-password=datahub
and 
kubectl create secret generic neo4j-secrets --from-literal=neo4j-password=datahub
it’s not in the same namespace. Now when I ran command 
helm install datahub datahub/datahub
I got an error, do you have any idea? Here’s the screenshoot from kubectl command and workloads
Update : Already succeed to open datahub locally
i
Hello Alvi, To confirm, did you successfully deploy DataHub in GCP?
a
Hello Pedro, Sure I already sucessfully deploy Datahub in GCP using kubernetes, here are the screenshoot. But some of my pods still pause and there is an error but it still works to open Datahub. Now I got the problem when ingesting the data, but I think I’ll open a new thread if still error and didn’t find the solutions.
i
The 
*job-template
cronJob Types are expected to be paused. Those are only meant to be triggered for one-time operations for very specific scenarios.
The 
nocode-migration-job
should not fail, can you share the logs for that pod?
a
Here’s my error log from one of the nocode-migration-job pod
nocode-migration-job-error-logs.txt
i
Are you able to re-run the job? Seems like a timing issue.
a
re-run the job did you mean re-run the ingestion?
i
No, re-running the no-code kubernetes job
a
I’m sorry I’m new regarding gcp and kubernetes, how to re-running one pod only?
Already re-deployed again but still error and no up for 
nocode-migration-job
Anyway I tried to run 
datahub-datahub-cleanup-job-template
to restart one job but suddenly error appears 
ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.7.2ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.7.2ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.7.2ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.7.2
i
There is no need to run the cleanup job.
And the No code migration job should be ok to not run either
a
Well I already re-deployed again and still 
nocode-migration-job
job still error. However I still can open datahub ui but when I tried to ingest the metada still error. One of error log in 
nocode-migration-job
is 
ERROR: Cannot connect to GMSat host datahub-datahub-gms port 8080. Make sure GMS is on the latest version and is running at that host before starting the migration.
I assume this error have something related when ingesting the data especially when you put this into your recipe yaml.
Copy code
sink:
    type: datahub-rest
    config:
        server: '<http://datahub-gms:8080>'
What do you think pedro?
i
The no code job has nothing to do with the ingestion. Where are you running the ingestion from?
The url must point to the kubernetes gms service
a
I’m running from UI for ingestion, so it must look like this? server : myIP:8080 ?
Anyway I got an error when ingesting and the error is
Copy code
{'version': ['Error: (psycopg2.OperationalError) connection to server at "{my_aws_host_port_name}" (10.100.2.209), port 5432 failed: Connection timed out\n\tIs the server running on that host and accepting TCP/IP connections?\n\n(Background on this error at: <https://sqlalche.me/e/14/e3q8)']>}
Do I need to configure something from kubernetes section or redshift?
i
If you are running the ingestion from the UI you should not need to configure the sink. Are you filling in the yaml form or through the UI?
The kubernetes pod in the datahub namespace called actions must have connectivity to whatever source you are trying to connect to. In this case 
10.100.2.209
a
Well yesterday I filled through UI still error, then tried to search something in here by using sink but still error. I see, let’s just say I need to whitelist my pod in my aws redshift?
i
Not the pod, you need to whitelist all Kubernetes node IPs in the AWS Redshift cluster. This is because the pod can be freely allocated to any of the nodes that you have in the Kuberentes cluster.
a
I see thank you for the insight. Anyway I tried different resources with BigQuery and successfully ingested with pods 
nocode-migration-job
still error. So I want to ask, what is the purpose of this pod?
i
From: https://datahubproject.io/docs/docker/datahub-upgrade/#supported-upgrades
Copy code
NoCodeDataMigration: Performs a series of pre-flight qualification checks and then migrates metadata_aspect table data to metadata_aspect_v2 table.
Since this is your first time deploying DataHub it’s not relevant
a
Let’s just say I want to deploy it on production, is it okay if I don’t wanna use this? Or is it mandatory?
i
It’s ok to not use
a
I see thank you so much for the information
4 Views
Open in Slack
PreviousNext
Powered by