Sorry for dropping in so many questions smile I tried some o DataHub #troubleshoot

Sorry for dropping in so many questions. :smile: I...

red-pizza-28006

11/02/2021, 3:05 PM

Sorry for dropping in so many questions. 😄 I tried some of the configs available in Snowflake ingestion as well. For e:g

profiling.limit

-> I started getting this error

Copy code

[SQL: CREATE OR REPLACE TEMPORARY TABLE ge_tmp_eee423e9 AS SELECT *
FROM src_salesforce."ORDER"
 LIMIT 2000

Without this, it seems to work fine. I also tried adding

Copy code

profile.turn_off_expensive_profiling_metrics

But for some reason it does not accept the profile param. I am on the latest datahub version 0.8.16.2

helpful-optician-78938

11/02/2021, 7:28 PM

Hi @red-pizza-28006, thanks for reporting. I'll take a look and get back to you.

red-pizza-28006

11/04/2021, 8:23 AM

any luck @helpful-optician-78938?

helpful-optician-78938

11/04/2021, 4:39 PM

Hi @red-pizza-28006, for turning off expensive profile metrics, is this how your recipe.yml looks?

Copy code

profiling:
        enabled: True
        turn_off_expensive_profiling_metrics: True

red-pizza-28006

11/04/2021, 4:40 PM

there is a profile key before that as well, right?

helpful-optician-78938

11/04/2021, 4:46 PM

nope, this sits directly under the

config:

helpful-optician-78938

11/04/2021, 4:52 PM

@red-pizza-28006, I am not able to reproduce this locally. What is the full error message? The query above is used by the great_expectations library and it is trying to create a temporary table. May be the role that you are using does not have sufficient privileges to create a table under the specific database+schema.

5 Views

Open in Slack

Previous Next