could someone share with push ingestion model, doc...
# ingestionb
brief-insurance-68141
10/26/2021, 6:15 PMcould someone share with push ingestion model, documents and any demo codes in the repo?
m
mammoth-bear-12532
10/26/2021, 6:29 PM
@brief-insurance-68141: take a look at the airflow integration here (https://github.com/linkedin/datahub/blob/master/metadata-ingestion/src/datahub_provider/_lineage_core.py)
b
brief-insurance-68141
10/26/2021, 6:34 PMthank you
brief-insurance-68141
10/26/2021, 6:43 PMIs there some way that data source (say mysql) push over rest api to datahub? SDK?
brief-insurance-68141
10/26/2021, 6:44 PMI figured out how to use pull method in datahub.
brief-insurance-68141
10/26/2021, 6:45 PMthere are some private databases lived only in private subnets, that datahub service can not talk.
m
mammoth-bear-12532
10/26/2021, 6:56 PM
you can just run the ingestion agent in the private subset and have it push the metadata out?
b
brief-insurance-68141
10/26/2021, 9:00 PMyes, that is one way. Does Datahub support that? could you share me with the document?
m
mammoth-bear-12532
10/26/2021, 9:51 PM
@brief-insurance-68141 that is the default thing that DataHub supports actually
mammoth-bear-12532
10/26/2021, 9:51 PM
so I’m not sure how to help you… you just need to find a host to run the
datahub ingestmammoth-bear-12532
10/26/2021, 9:52 PM
you can either run it bare (using pip install) or run it using the docker container
mammoth-bear-12532
10/26/2021, 9:52 PM
I’m assuming you have read this page (https://datahubproject.io/docs/metadata-ingestion/)
b
brief-insurance-68141
10/27/2021, 1:20 AMgot it. thank you. that is the page. I read it, but not detailedly.
m
mammoth-bear-12532
10/27/2021, 1:51 AM
great.. let us know if we need to improve how we are describing it!
b
brief-insurance-68141
10/27/2021, 6:07 PM😁