Hello everyone! Just deployed DataHub on K8s cluster and now looking into how to install various plugins (I new to both DataHub and K8s). With my local docker deployment I used pip install but I doubt that is the correct approach with the K8s. How should I do that? Thank you in advance!

Please see https://datahubproject.io/docs/how/auth/add-users/

@square-activity-64562 I think you meant to post this on another thread 🙂 Where are you running the CLI commands from? It's entirely possible that this is still the right approach, it just depends on if you plan on connection to the K8s cluster from your local machine or from somewhere else. It's also possible to do the ingestion through the UI: https://datahubproject.io/docs/ui-ingestion/ instead of using the CLI.

Hi @orange-night-91387 I would like to use the UI for ingestion but as far as I understand I have to first install the respective plugins if not already installed. Are you suggesting to install the DataHub CLI on my local machine (which I use to run the kubectl commands) and somehow install the plugins with it on the K8s? Do you have any example of the commands I need to use to connect to the K8s DataHub deployment using the CLI? Or shall I use the CLI to connect to the pod running GMS (or another pod) and then install the plugins in the GMS container? Many thanks in advance for your help!

The plugins need to be present on the machine connecting to the data source which is the machine where the Python commands are executing. The UI ingestion executes on a Docker image we have preloaded with the necessary plugins (some sources may have issues like Oracle which there is an open PR to address). For using the CLI they just need to be present on the machine running the CLI as the connection occurs there and then sends MCPs to GMS. You'll just have to change the recipe accordingly

Hi @orange-night-91387 I hope I am starting to understand - just to check with you that this is what I am supposed to do: • I installed DataHub CLI locally • Now I need to use datahub init <datahub-gms> to tell the CLI where the DataHub gms is located, is this correct? IN order to do that I should generate access token first to fill in the TOKEN value, right? And then I can use datahub ingest <some recipe.yml>, is that correct? ◦ When I try to generate access token from the UI it says that 'Token based authentication is currently disabled'. I am trying to enable it as explained here https://datahubproject.io/docs/introducing-metadata-service-authentication/#configuring-metadata-service-authentication ◦ Do I need to change the helm charts in order to do that? Is this the correct approach? Many thanks!

1. Yep! You'll run

datahub init

and supply it with the location of GMS 2. You may or may not need to do this depending on your set-up. If you do not currently have Metadata Service Authentication set up, your GMS API should not require a token to access, only if you only have access to route the requests through the Frontend proxy. 3. Yes, if you've deployed with k8s, you'll need to configure the environment by modifying the helm charts. Hope this helps!

Hi @orange-night-91387! Yes, I managed to run CLI ingestion using the GMS API (and the token was not needed). Many thanks for the help!

@purple-soccer-81736 🙂