We started using volume anomaly detection with a

timestamp_column

and we notice that the queries generated by the test filter far far further back than the training period. how can we limit that? we have some large tables and want to make it as efficient as possible Ideally since the tables are partitioned by the timestamp column we want to only filter on the duration of the test+detection

This is a known performance issue with Elementary's volume anomaly tests, and unfortunately the knowledge sources indicate it hasn't been fully resolved. The Root Cause Elementary's volume tests first execute a broad scan:

select timestamp_column
from table
where timestamp_column >= training_period

...and only then calculate buckets and counts. This means it doesn't leverage partition pruning even when your

timestamp_column

is the partition key. [Slack thread] What You Can Do Today 1. Reduce the training_period The most direct lever — a smaller training window means a smaller scan:

models:
  - name: your_model
    tests:
      - elementary.volume_anomalies:
          timestamp_column: your_date_column
          training_period:
            period: day
            count: 7   # reduce from the default 14 days
          detection_period:
            period: day
            count: 2

[training_period docs] 2. Use where_expression to add a partition filter You can add a

where_expression

to explicitly filter on your partition column, which should allow your warehouse to use partition pruning. This is the workaround recommended in the community for tables where the

timestamp_column

is the partition key. [Slack thread] 3. Pre-aggregate into a smaller table One community-validated workaround is to create a pre-aggregated table (e.g.,

timestamp | row_count

per bucket) and run the Elementary test against that instead of the raw large table. This has been confirmed to work well in practice. [Slack thread] The Underlying Issue The community has raised this as a feature request — the ideal fix would be to calculate the time buckets first and then query the table directly per bucket, which would allow partition metadata to be used and reduce runtime dramatically. Elementary acknowledged this as a known issue in their backlog, but as of the last community updates in the knowledge sources, it had not yet been resolved. [GitHub issue] In short: reducing

training_period

and adding a

where_expression

are your best immediate options. The pre-aggregation approach is the most robust workaround if query performance is critical. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.