https://linen.dev logo
Join Slack
Powered by
<@U088WCP3B4P> Is there an elementary test that wo...
# ask-elementary-ai
d
@kapa.ai Is there an elementary test that works like dbt_utils.expression_is_true
k
A message from kapa.ai
There isn’t an Elementary-native test that’s explicitly documented as a drop‑in replacement for 
dbt_utils.expression_is_true
. The knowledge sources consistently show 
expression_is_true
as coming from 
dbt_utils
and being used alongside Elementary, not replaced by it. [[Data quality dimensions](https://docs.elementary-data.com/cloud/best-practices/detection-and-coverage#data-quality-dimensions-framework)] Elementary’s guidance is: - Use 
dbt_utils.expression_is_true
(or other dbt/dbt-expectations tests) for fixed business rules and constraints. [[Data quality dimensions](https://docs.elementary-data.com/cloud/best-practices/detection-and-coverage#data-quality-dimensions-framework)] - Use Elementary’s own tests mainly for anomaly detection (volume, freshness, column anomalies, etc.) and schema monitoring. [[dbt tests in Cloud](https://docs.elementary-data.com/cloud/features/data-tests/dbt-tests#page-title)] Elementary treats all dbt tests (including 
dbt_utils.expression_is_true
) as first‑class: it collects their results, shows them in the UI, and can alert on them. [[dbt tests in Cloud](https://docs.elementary-data.com/cloud/features/data-tests/dbt-tests#page-title)] So if you want “`expression_is_true`‑like” behavior, the recommended approach is to keep using 
dbt_utils.expression_is_true
within your dbt project and let Elementary observe and alert on those test results.
d
@kapa.ai if I want to set the meta config to add owner, channel, and subscribers, would elementary alert on 
dbt_utils.expression_is_true
?
k
A message from kapa.ai
Yes. Elementary will alert on 
dbt_utils.expression_is_true
tests, and it will use the 
meta
configuration (including 
owner
, 
channel
, and 
subscribers
) the same way it does for other dbt tests. From the docs: - Elementary can send alerts for failures/warnings of dbt tests, not just its own tests. [[Elementary alerts](https://docs.elementary-data.com/oss/guides/alerts/elementary-alerts#page-title)] - Alert properties like 
owner
, 
subscribers
, 
channel
, and others are configured under 
meta
and apply to tests as well. [[Alert properties](https://docs.elementary-data.com/oss/guides/alerts/alerts-configuration#alert-properties-in-yml-files); [Alerts distribution](https://docs.elementary-data.com/oss/guides/alerts/alerts-configuration#alerts-distribution)] A minimal example for a generic test (including 
dbt_utils.expression_is_true
) in a 
.yml
file:
Copy code
yml models:   - name: my_model     data_tests:       - dbt_utils.expression_is_true:           name: my_business_rule           expression: "some_column > 0"           config:             meta:               owner: "@jessica.jones"               subscribers: ["@joe.joseph"]               channel: data_ops
As long as: 1. The test runs and fails (or warns) in 
dbt test
, and 2. You run 
edr monitor
(OSS) or have Cloud sync enabled, Elementary will pick up that test result and send alerts enriched with the 
owner
, 
subscribers
, and 
channel
from 
meta
. [[Quickstart alerts](https://docs.elementary-data.com/oss/quickstart/quickstart-alerts#page-title); [Alert properties](https://docs.elementary-data.com/oss/guides/alerts/alerts-configuration#alert-properties-in-yml-files)]
d
@kapa.ai when using column_anomalies, should configurations be stored under config: or argument: if using dbt version 1.11
k
A message from kapa.ai
For dbt 1.11 (Fusion), 
column_anomalies
must use the new 
arguments
 / 
config
 split: • Test arguments (what the macro uses: 
column_anomalies
, 
timestamp_column
, 
time_bucket
, etc.) go under 
arguments
. • Test configuration (dbt-level settings like 
severity
, 
tags
, 
meta
) go under 
config
. [dbt Fusion OSS; column_anomalies config] From the Elementary docs for Fusion-compatible syntax: [dbt Fusion OSS] > 1. Test arguments must be encapsulated under an 
arguments
field. > 2. Configuration fields such as 
meta
, 
tags
or 
severity
must be encapsulated under a 
config
field. And the 
column_anomalies
page shows the Fusion-style structure: [column_anomalies config]
Copy code
models:
  - name: my_model
    columns:
      - name: my_column
        data_tests:
          - elementary.column_anomalies:
              arguments:
                column_anomalies: [average, null_percent]
                timestamp_column: event_ts
                time_bucket:
                  period: day
                  count: 1
                anomaly_sensitivity: 3
              config:
                severity: warn
                tags: ["elementary"]
So for dbt 1.11: • Put all anomaly parameters under 
arguments
. • Put severity/tags/meta under 
config
. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai With dbt version 1.11, do we place elementary configurations in config: or arguments: or meta:? Why are elementary channel, owner etc placed under meta in config?
k
A message from kapa.ai
For dbt 1.11 (Fusion), you need to separate test arguments from test configuration: • 
arguments:
 → what the Elementary test logically uses (e.g. 
timestamp_column
, 
time_bucket
, 
column_anomalies
, 
dimensions
, etc.). • 
config:
 → dbt test configuration (e.g. 
severity
, 
tags
, 
meta
). [dbt Fusion OSS] Elementary’s alert-related settings (
owner
, 
channel
, 
subscribers
, 
alert_fields
, etc.) are alert metadata, not test logic, so they live under 
meta
inside 
config
. Elementary reads these from 
meta
on the resource (model/source/test) according to its precedence rules. [Alert properties] Putting it together for a Fusion-style Elementary test:
Copy code
models:
  - name: my_model
    columns:
      - name: my_column
        data_tests:
          - elementary.column_anomalies:
              arguments:
                timestamp_column: event_ts
                time_bucket:
                  period: day
                  count: 1
                column_anomalies: [average]
              config:
                severity: warn
                tags: ["elementary"]
                meta:
                  owner: "@jessica.jones"
                  channel: data_ops
                  subscribers: ["@joe.joseph"]
So: • Elementary-specific anomaly params → 
arguments:
• dbt/alert config (including Elementary 
owner
, 
channel
, 
subscribers
, etc.) → 
config: meta: ...
because Elementary is documented to look for these under 
meta
. [Alert properties; dbt Fusion OSS] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai the elementary report generated for an anomaly detection test is only showing 1 day instead of the 14 days of being tested. What could be happening? Is there a known bug for this?
k
There are a few known causes for the Elementary report showing only 1 day of data instead of the full training/detection period. Here's what the knowledge sources point to: 1. days_back CLI flag is too low When generating the report, the 
--days-back
flag controls how much history is shown in the UI. If you're running something like 
edr report -d 7
but your training period is 14 days, the report will only display data within that window. [Anomaly troubleshooting] Try:
Copy code
edr report --days-back 14
or even higher if needed. 2. Training period was recently changed If you changed the 
training_period
(or 
days_back
var) after tests had already run, the metrics table won't automatically backfill. You need to do a full refresh: [Anomaly troubleshooting]
Copy code
dbt run --select data_monitoring_metrics --full-refresh
Then re-run your Elementary tests. 3. Not enough test runs yet (no timestamp_column) If your test has no 
timestamp_column
, the training set builds up over multiple test runs (one data point per run). For a 14-day training period, you'd need 14 separate runs on different days. If the test was recently set up, there may simply not be enough history yet. [Anomaly troubleshooting] 4. Known version-related bug There have been reports of historical data not displaying correctly in the dashboard even when the underlying tables had the data. This was confirmed as a bug and fixed in a later CLI version. After upgrading, the graph should populate correctly as new runs are added. [Missing historical data] 5. Report display issues (browser) One user reported a 1 GB report that showed "No test results" in Chrome but worked in Safari. If your report is very large (e.g., hourly buckets over a long period), this could be a factor. [Report display] Recommended first steps: 1. Re-run with 
edr report --days-back 14
(or higher). 2. Check if 
data_monitoring_metrics
actually has 14 days of data for your table. 3. If you recently changed 
training_period
, do a full refresh of 
data_monitoring_metrics
. 4. Ensure you're on the latest Elementary CLI and dbt package versions. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.
31 Views
Open in Slack
PreviousNext
Powered by