@kapa.ai Can elementary freshness anomaly detection be used on sources - i.e. things not updated by dbt - in a way that makes sense or does it only make sense in the context of building models?

Yes, Elementary freshness anomaly detection can be used directly on dbt sources (including ones not updated by dbt), and it’s explicitly designed to work on both sources and models. 1. Supported on sources and models • Elementary’s

freshness_anomalies

test is a regular dbt test and “works the same on sources and on models.” [Freshness on both] • You add it in your

sources:

YAML just like you would under

models:

. [Add tests to sources; Source YAML example] Example:

version: 2

sources:
  - name: raw_app
    schema: raw_app
    tables:
      - name: events
        tests:
          - elementary.freshness_anomalies:
              timestamp_column: event_time
              time_bucket:
                period: day

You run it with

dbt test

(not

dbt source freshness

). [How to run; dbt test vs source freshness] 2. When it “makes sense” on sources It makes sense if your source table: • Has a timestamp column that reflects when data arrives (e.g. ingestion or ETL time). [Config on sources] • Is normally updated on a regular cadence (hourly, daily, etc.), so Elementary can learn a baseline and detect when it slows or stops. [Behavior explanation] In that case, using

freshness_anomalies

on the source is often more valuable than only on downstream models, because you catch issues as close to ingestion as possible. 3. When it does not behave like a simple “staleness” check On both sources and models,

freshness_anomalies

is anomaly-based, not a fixed “fail if older than X days” rule: • It buckets data by time and compares recent buckets to historical ones. [How it works] • If a source has been stale for the entire training window (e.g. no updates for a month and your history is also a month of “no updates”), the test will likely pass, because “no updates” is the learned normal. [Stale table example; Similar case] So: • Use

freshness_anomalies

on sources when you want to detect changes in update behavior (e.g. a daily source suddenly stops or slows). • If you need a strict “fail if no data for N days” on a source, you should use

dbt source freshness

or a custom/generic test instead of relying on

freshness_anomalies

. [Recommendation for staleness; dbt source freshness vs Elementary] Summary • Yes, it absolutely makes sense to run Elementary freshness anomaly detection directly on sources, not just on dbt-built models. • It’s best for sources that normally update regularly and where you care about anomalous delays, not just absolute age. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.