https://datahubproject.io logo
Join Slack
Powered by
I have deployed DataHub with Kubernetes into Azure...
# ingestion
f
I have deployed DataHub with Kubernetes into Azure Kubernetes Service, following the deployment guide in the docs (adapted a bit for azure). I need to install a plugin into my AKS cluster (specifically “sqlalchemy.dialects:databricks.pyhive”). How do I do this? I am getting the following error telling me I need this plugin:
Copy code
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:databricks.pyhive
After some additional digging. It looks like the pip command that are run in the execution details are run on the ‘datahub-acryl-datahub-actions’ pod
I can exec a bash shell into this pod, but I suspect that is not the correct way to run 
pip install
for this python package
s
Currently we don't support making changes to the actions pod used for UI based ingestion
f
Any advice on how to resolve this error?
s
@big-carpet-38439 ^
b
Is this only happening when you try to ingest via the UI?
Or when you ingest using the 
datahub ingest
command
f
I have only tried ingest using the UI.
This is the recipe I am using. It is based off the example recipe in the hive source docs.
cleaned_recipe.yml
b
Right - so I'd recommend to use the 
datahub
to push this since the UI doesn't support this dependency right now. It will be simpler than mounting those dependencies into DataHub's container
f
Tried the CLI. It appears to work better and get past this issue. Thanks
plus1 1
Open in Slack
PreviousNext
Powered by