I am having a query is regarding elementary python custom test. So if my model is incremental and if I run a custom test, it will run the test on new chunk of data or whole dataset? Example model from documentation:

{% macro compare_tables(args) %}
import pandas as pd

def test(model_df, ref, session):
    original_table = model_df.toPandas().drop_duplicates()
    other_table = ref('{{ args.other_table }}').toPandas().drop_duplicates()
    return pd.concat([original_table, other_table]).drop_duplicates(keep=False)
{% endmacro %}

Thanks in advance!

Hey @Anil, tests (sql or python) runs on the whole data of the model (even if it is incremental) 🙂

ok, thanks for quick confirmation. Also can you help me with this error in setting up slack alerts:

File "C:\Projects\data_pipeline\venv_dbt\Lib\site-packages\elementary\monitor\data_monitoring\alerts\data_monitoring_alerts.py", line 131, in _get_suppressed_alerts
    (current_time_utc - last_sent_time).total_seconds() / 3600
     ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
TypeError: can't subtract offset-naive and offset-aware datetimes

Sure 🙂 can I ask which version of elementary you are using?

0.14.0

Cool 🙂 So this bug was fixed at elementary 0.14.1 Care to upgrade and see if this is resolved?

Sure, will try. Thanks again 🙂

Happy to hear that it worked! About the second issue, this is not something that is known. Can you share your dbt.log with us, and open an issue as well?

Here is a link to open an issue: https://github.com/elementary-data/elementary/issues/new/choose 🙂G

Attaching truncated dbt logs:

[0m16:59:00.062877 [debug] [MainThread]: Writing injected SQL for node "operation.elementary.elementary-on-run-end-0"
[0m16:59:00.062877 [info ] [MainThread]: 1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
[0m16:59:00.062877 [info ] [MainThread]: 1 of 1 OK hook: elementary.on-run-end.0 ........................................ [[32mOK[0m in 0.00s]
[0m16:59:00.062877 [info ] [MainThread]: 
[0m16:59:00.078404 [debug] [MainThread]: Connection 'master' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'list_data-dev__dev_transformed_fb_ads' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'list_data-dev__dev_transformed_elementary' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__ad_sets_raw_json_data_id.626c335db0' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__ads_raw_json_data_id.d90f492282' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__ads_insights_action_conversion_device_raw_json_data_id.cbc379487f' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__ads_insights_raw_json_data_id.7e916ffb04' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__ad_account_raw_json_data_id.e50e89ee25' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__campaigns_raw_json_data_id.744fbb1502' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_stg_fb_marketing__activities_raw_json_data_id.a949f5d854' was properly closed.
[0m16:59:00.078404 [debug] [MainThread]: Connection 'test.dbt_big_query_datadev.unique_fct_ads_daywise_report_ad_name.0a679832dd' was properly closed.
[0m16:59:00.078404 [info ] [MainThread]: 
[0m16:59:00.078404 [info ] [MainThread]: Finished running 118 tests, 2 hooks in 0 hours 1 minutes and 26.36 seconds (86.36s).
[0m16:59:00.094132 [debug] [MainThread]: Command end result
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [info ] [MainThread]: [31mCompleted with 4 errors and 0 warnings:[0m
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [error] [MainThread]:   Internal Error
  popped ['materialization'], expected macro.elementary.test_python
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [error] [MainThread]:   Internal Error
  popped ['materialization'], expected macro.elementary.test_python
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [error] [MainThread]: [31mFailure in test elementary_freshness_anomalies_dim_ads_insights_day__1__day__1__day__30 (models\marts\core\fb_marketing\fb_marketing_marts.yml)[0m
[0m16:59:00.217635 [error] [MainThread]:   Got 2 results, configured to fail if != 0
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [info ] [MainThread]:   compiled Code at target\compiled\dbt_big_query_datadev\models\marts\core\fb_marketing\fb_marketing_marts.yml\elementary_freshness_anomalies_d54039c4d7b66438d0c09b54f082ba63.sql
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.217635 [error] [MainThread]: [31mFailure in test unique_fct_ads_daywise_report_ad_name (models\marts\analytics\fb_marketing\fb_marketing_analytics.yml)[0m
[0m16:59:00.217635 [error] [MainThread]:   Got 520 results, configured to fail if != 0
[0m16:59:00.217635 [info ] [MainThread]: 
[0m16:59:00.233304 [info ] [MainThread]:   compiled Code at target\compiled\dbt_big_query_datadev\models\marts\analytics\fb_marketing\fb_marketing_analytics.yml\unique_fct_ads_daywise_report_ad_name.sql
[0m16:59:00.233304 [info ] [MainThread]: 
[0m16:59:00.233304 [info ] [MainThread]: Done. PASS=114 WARN=0 ERROR=4 SKIP=0 TOTAL=118
[0m16:59:00.233304 [debug] [MainThread]: Command `dbt test` failed at 16:59:00.233304 after 87.76 seconds
[0m16:59:00.233304 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020C1884A410>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020C184E6C50>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000020C18CF1B90>]}
[0m16:59:00.233304 [debug] [MainThread]: Flushing usage events

Hey @Anil, this issue happens when there is an exception in the python code, sadly, due to a limitation in the way that

dbt

allows for python code execution, the exception itself cannot be retrieved and displayed in the command output, so you would need to check it in your data warehouse