This message was deleted.
# troubleshooting
s
This message was deleted.
k
I am not aware of any druid snowflake connectors.
s
I haven't tried this. But in principle you should be able to use JDBC. I think the content of the EXTERN clause should look like the sql input source. e.g.:
Copy code
"type": "sql",
        "database": {
            "type": "mysql",
            "connectorConfig": {
                "connectURI": "jdbc:<mysql://host>:port/schema",
                "user": "user",
                "password": "password"
            }
        },
        "sqls": ["SELECT * FROM table1 WHERE timestamp BETWEEN '2013-01-01 00:00:00' AND '2013-01-01 11:59:59'", "SELECT * FROM table2 WHERE timestamp BETWEEN '2013-01-01 00:00:00' AND '2013-01-01 11:59:59'"]
and you will need to have the Snowflake JDBC driver in the classpath.... but upon further reading it does seem like you will also need an extension that enables the
connectorConfig
for a new
"database": { "type": "snowflake"....}
, so I don't think this is available out of the box.
g
note the SQL-based ingest supports all the same input sources as native; they are listed here https://druid.apache.org/docs/latest/ingestion/native-batch-input-sources.html. you can provide them all in EXTERN
for reading larger amount of data from snowflake i think it'd be best to export first, perhaps using data unload https://docs.snowflake.com/en/user-guide/data-unload-overview.html
then we can read the exported data using input source like
s3
or
gcs
depending on your cloud
it's nice since it's automatically done in parallel