The all columns anomalie s test has excluded some columns is Elementary Community #community-support

The *all_columns_anomalie*s test has excluded some...

Damian Eralio

10/13/2023, 2:04 PM

The *all_columns_anomalie*s test has excluded some columns, is this a bug? setup:

- name: customer_phone

tests:

- elementary.all_columns_anomalies:

tags: ["sources"]

column_anomalies: ["null_count", "null_percent", "missing_count"]

exclude_prefix: ["_dbt", "_fivetran"]

table (see snowflake table columns listed below): results (see elementary dashboard results below): the columns

normalized

and

type

were excluded for some reason

👀 1

Damian Eralio

10/13/2023, 3:12 PM

After some more testing. This has to be a bug. commenting out the

exclude_prefix

parameter does test all columns. So the bug is that:

exclude_prefix: ["_dbt", "_fivetran"]

is excluding columns

type

normalized

_fivetran_deleted

_fivetran_synced

Damian Eralio

10/13/2023, 5:03 PM

coming back to finish up this thread. I will open a ticket in the github repo. But the workaround for now seems to be using

exclude_regexp

as it is working well

exclude_regexp: "^(_dbt|_fivetran).*"

Ella Katz

10/18/2023, 2:39 PM

Hi @Damian Eralio, Apologies for the late response. We only allow a single exclude prefix, when using multiple prefixes it's required to use the regexp like you did. What the prefix does is create a regexp using the prefix given. If you try

exclude_prefix: "_fivetran"

I believe it should work for you.

🙌 1

Open in Slack

Previous Next