https://datahubproject.io logo
#ingestion
Title
# ingestion
b

best-umbrella-24804

06/14/2022, 12:25 AM
Hello, I have a recipe for snowflake usage that looks like this
Copy code
source:
    type: snowflake-usage
    config:
        env: DEV
        host_port: <http://xxxx.snowflakecomputing.com|xxxx.snowflakecomputing.com>
        warehouse: DEVELOPER_X_SMALL
        username: DATAHUB_DEV_USER
        password: '${SNOWFLAKE_DEV_PASSWORD}'
        role: DATAHUB_DEV_ACCESS
        top_n_queries: 10
sink:
    type: datahub-rest
    config:
        server: '<http://xxxxxx>'
We are finding that there are 1600 instances of the following error in our logs
Copy code
'2022-06-13 23:52:41.644712 [exec_id=66448a5e-1f15-406f-8ee7-700ed32f427b] INFO: stdout=[2022-06-13 23:52:37,635] WARNING  '
"{datahub.ingestion.source.usage.snowflake_usage:93} - usage => Failed to parse usage line {'query_start_time': datetime.datetime(2022, "
'6, 12, 4, 33, 7, 869000, tzinfo=datetime.timezone.utc), \'query_text\': "INSERT INTO xxxxx( ..... )            '
'validation error for SnowflakeJoinedAccessEvent\n'
'email\n'
'  none is not an allowed value (type=type_error.none.not_allowed)\n'
Any idea what is going on here?
l

loud-island-88694

06/14/2022, 6:46 AM
@square-activity-64562 ^ is this a recent regression?
s

square-activity-64562

06/15/2022, 4:41 AM
@best-umbrella-24804 You are not using
email_domain
in your config. Can you please add that. Request you to share the CLI version and server version that you are using
p

prehistoric-salesclerk-23462

07/05/2022, 2:56 PM
@best-umbrella-24804 were you able to fix this issue?
b

best-umbrella-24804

07/06/2022, 12:33 AM
No I just switched profiling off
p

prehistoric-salesclerk-23462

07/06/2022, 7:49 AM
@best-umbrella-24804 I fixed it by adding
email_domain
in the ingestion config as suggested by @square-activity-64562
👍 1
b

best-umbrella-24804

07/06/2022, 11:46 PM
ah for some reason that didnt register thanks