Hi all. I am brand new to Datahub. I'm trying my v...
# ingestion
n
Hi all. I am brand new to Datahub. I'm trying my very first connection. I'm trying to configure Snowflake to import a schema and provide definitions. I simply keep getting a connection refused error whenever I try to connect. I think the challenge is with the
host_port
piece but I can't figure out what is wrong. Whenever I connect with Python I seem to have to provide azure credentials as part of the connection string.
Copy code
source:
    type: snowflake
    config:
        host_port: [snowflakeaccount].[azure-region].<http://azure.snowflakecomputing.com|azure.snowflakecomputing.com>
        warehouse: *****
        username: *****
        password: *****
        role: ****
sink:
    type: datahub-rest
    config:
        server: '<http://localhost:9002/api/gms>'
f
the server is http://localhost:8080
n
Copy code
source:
    type: snowflake
    config:
        host_port: [snowflakeaccount].[azure-region].<http://azure.snowflakecomputing.com|azure.snowflakecomputing.com>
        warehouse: *****
        username: *****
        password: *****
        role: ****
sink:
    type: datahub-rest
    config:
        server: '<http://localhost:8080>'
This failed with a "connection refused" error./
l
Hi @numerous-holiday-52504, chatting with the team now - try removing
Copy code
.[azure-region].<http://azure.snowflakecomputing.com|azure.snowflakecomputing.com>
and only pass
Copy code
host_port: [snowflakeaccount]
cc: @big-carpet-38439
n
How does the tool know it's hosted on AWS, Azure or google without that piece?
This has all the options that are possible, and none of them seem to work LINK
m
you would have the snowflake_account.region i.e.
Copy code
<snowflake_account>.ap-southeast-2
@gray-river-37120 you may want to try running
datahub --debug ingest -c <recipe>
to see what sort of details datahub send to the host.
g
Thanks @modern-artist-55754. Tried it but I get an error that
{datahub.entrypoints:165} - snowflake is disabled; try running: pip install 'acryl-datahub[snowflake]'
Trying that install also produces an error. But it seems odd to me that this would be the case when doing it through the datahub UI works up until the authentication (which would imply to me that
snowflake
is, in fact, enabled.
m
@gray-river-37120 oh so you are missing the
acryl-datahub[snowflake]
python package, it makes sense that it works up till the authentication, because that’s when it try to use the snowflake library. You need to get into the datahub-action pod (if you run k8s) and install the package. I run my own datahub container with all the packages installed