Hi guys, I am trying to ingest snowflake metadata ...
# ingestion
g
Hi guys, I am trying to ingest snowflake metadata into Datahub but getting this error - pkg_resources.ContextualVersionConflict: (certifi 2021.5.30 (/usr/local/lib/python3.8/site-packages), Requirement.parse('certifi<2021.0.0'), {'snowflake-connector-python'}) My snowflake recipe structure looks like this -
source:
type: "snowflake" config: username: user password: "password" host_port: port database: database warehouse: warehouse table_pattern: allow: - "DATABASE.SCHEMA.TABLE_NAME" sink: type: "datahub-rest" config: server: 'server_url'
b
@mammoth-bear-12532 @witty-state-99511 Any ideas?
w
seems like a snowflake connector dependency issue
b
yes, it is
g
How to solve this?
m
@gorgeous-glass-57878 do you have a pip venv just dedicated to datahub?
@gorgeous-glass-57878 does this happen even in a dedicated venv?
g
Let me check and get back to you. We used the same environment for different database ingestion, and it worked fine.
The error is coming in the pod. The issue seems to be with the image we are using. Can anyone help to resolve this?
b
which image and version are you using?
g
datahub-ingestion-cron: enabled: false image: repository: linkedin/datahub-ingestion tag: "latest"
e
Can you try using tag: “head” ?
or the same version you are using for everything else like “v0.8.11”