Hi datahub! I have some trouble in datahub system....
# ingestion
s
Hi datahub! I have some trouble in datahub system. Can I get help? I want to ingest dataset of postgreSQL. and I created ingestion source with web ui like below picture. and I decide the schedule as ’00 03 * * *’. But I initially did run manually to see if the data ingestion was good, but it doesn’t sink normally. What is the reason and how can I solve it? It was possible to manually sink through CLI on my EKS server.
s
What do you mean by "but it doesn’t sink normally". If you wish to manually do ingestion through CLI it is possible. You need to point the sink to the GMS pod. You can use
kubectl
to port-forward GMS port 8080 to your localhost and use CLI to sink to Datahub through GMS. Or you can schedule it through one of your existing tools as explained in https://datahubproject.io/docs/metadata-ingestion/schedule_docs/intro/
But ideally we would like to get it done through the UI
Can you click on the
+
button and share a screenshot? It is not clear what problem you are facing with UI based ingestion
✔️ 1
s
s
Can you plese Can you please give output of
kubectl get pods
for the pods running? Mainly there would be a pod which will have
actions
in it which we want to see status of. Any logs from it would be helpful
s
This is the screenshot when I run the command
kubectl get pods
. and I wanted to tell you that even if I pressed the EXECUTE button on the second attached picture to manually ingest the recipe file on datahub web ui, the ingest did not proceed.
s
There is no actions pod. You are probably using an older helm chart version. Please update the helm chart version.
s
What is action pod? I re-installed helm chart (datahub, prerequisites). but the types of pods are the same.
s
Something like
dh-acryl-datahub-actions-7bc4b6ff9c-66nmg
Refers to this https://github.com/acryldata/datahub-helm/tree/master/charts/datahub/subcharts/acryl-datahub-actions subchart
s
@square-activity-64562 Thank you for your answer
@square-activity-64562 How can I add the action pod? there is no pod of ‘acryl datahub action’ or I couldn’t find it. https://artifacthub.io/packages/helm/datahub/datahub Can I get some info to add the action pod?
s
The chart contains the subchart https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/Chart.yaml. Maybe the README was not updated You can see the values in https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/quickstart-values.yaml and use these values.
👍 1