Hi, I have setup datahub on our eks aws cluster an...
# ingestion
c
Hi, I have setup datahub on our eks aws cluster and exposed through ingress datahub-frontend en datahub-gms. When I try to run an ingest of a sample mce:
Copy code
datahub ingest -c ./examples/recipes/file_to_datahub_rest.yml
I receive a 500 error: Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘datahub.metadata_aspect_v2’ doesn’t exist. If I create the table manually and run the ingest again it does seem to work:
Copy code
datahub ingest -c ./examples/recipes/file_to_datahub_rest.yml
[2021-06-10 14:02:29,579] INFO     {datahub.entrypoints:68} - Using config: {'source': {'type': 'file', 'config': {'filename': './examples/mce_files/single_mce.json'}}, 'sink': {'type': 'datahub-rest', 'config': {'server': '<https://datahub-gms-test.data.dpgmedia.cloud>'}}}
[2021-06-10 14:02:29,876] INFO     {datahub.ingestion.run.pipeline:44} - sink wrote workunit file://./examples/mce_files/single_mce.json:0

Source report:
{'failures': {}, 'warnings': {}, 'workunit_ids': ['file://./examples/mce_files/single_mce.json:0'], 'workunits_produced': 1}
Sink report:
{'failures': [], 'records_written': 1, 'warnings': []}

Pipeline finished successfully
But I don’t see any rows in the
metadata_aspect_v2
table? Ingesting the same to datahub hosted in a local docker environment does work: entry is added to that table. Any ideas?
I managed to get it working by rerunning the helm install, which, I guess reran some of the setup containers.
b
Great to hear Peter. Let us know if you run into anything else... cc. @early-lamp-41924