A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi,
This might be a pretty basic question, but I'm trying to ingest MSSQL from the UI, but It require pyodbc. I have a deployment via the official helm chart.
Where/how can I actually install it?
I mean which subchart actually takes care of ingestion?

Our mssql source doesn’t need pyodbc unless you require encryption

To answer your question, ingestion configured through the UI actually runs in the datahub-actions pod

Yea, i need the encryption :slightly_smiling_face:
Thank you very much!

<@U01GZEETMEZ>
But it seems that even if i install custom requirement (pyodbc) into a custom docker image and use that, it's not picked up.
It seems the task installs it's own requirements.
How should i therefore install custom packages into a Ingestion run?

Unfortunately we don’t have a great way to do that yet - I’d recommend running ingestion from the CLI in the short run

I see <@U01GZEETMEZ> thank you very much for you help.
And by cli do you mean setting up linkedin/datahub-ingestion in my deployment or running cli on my dev machine?

I’d recommend running the cli on your dev machine. Checkout these docs <https://datahubproject.io/docs/cli> and <https://datahubproject.io/docs/metadata-ingestion> for details

Hmmm, that's not really feasible for us.
Alrighty, thanks for your help :slightly_smiling_face:

You can also use the linkedin/datahub-ingestion image to run ingestion, but you’ll need to build a custom image on top because the default one doesn’t include the MSSQL ODBC driver

thats fine :slightly_smiling_face: . Coooool, thanks again

managed :slightly_smiling_face: Once again, thanks for the guidance