Hi there, I'm trying to do a Snowflake ingestion. ...
# ingestionw
wooden-notebook-33413
10/21/2021, 1:34 PMHi there, I'm trying to do a Snowflake ingestion. I've run the example to pull the Top N Queries but now I want to capture the metadata of a specific DB while ensuring that I exclude one with a similar name.
I've added the following to my yml
database_pattern:
allow: "THISDB"
database_pattern:
ignoreCase: "NOTTHISDB"
I keep getting the following error:
1 validation error for SnowflakeUsageConfig
database_pattern
extra fields not permitted (type=value_error.extra)
✅ 1
p
polite-flower-25924
10/21/2021, 1:53 PMHey @wooden-notebook-33413, if I were you, I would try with the following structure.
Copy code
database_pattern:
allow:
- "THISDB"
ignoreCase: "NOTTHISDB"polite-flower-25924
10/21/2021, 1:54 PMallowdatabase_pattern:w
wooden-notebook-33413
10/21/2021, 1:56 PMI changed it to this :
database_pattern:
allow:
- "THISDB"
and i get the same error
p
polite-flower-25924
10/21/2021, 2:09 PMcould you please share the full configuration with masking the credentials and url
w
wooden-notebook-33413
10/21/2021, 2:16 PMsource:
type: snowflake-usage
config:
# Coordinates
host_port: <url>
warehouse: "<warehouse>"
# Credentials
username: <user>
password: <pass>
role: "ACCOUNTADMIN"
# Options
# top_n_queries: 10
database_pattern:
allow:
- "THISDB"
# ignoreCase: "NOTTHISDB"
sink:
# sink configs
type: "datahub-rest"
config:
server: "http://localhost:8080"
wooden-notebook-33413
10/21/2021, 3:23 PMfound the issue. I was using snowflake-usage instead of snowflake. thanks
🙌 2