<@U088WCP3B4P> I am trying to collect metrics usin...
# ask-elementary-ai
c
@kapa.ai I am trying to collect metrics using volume_anomalies all the way back to the earliest point I have data. Is it proper to use training_period for this backfill? Do I need to only run tests once in the environment to backfill that training_period and then I can set training_period back to a more normal range?
k
A message from kapa.ai
To backfill metrics for
volume_anomalies
all the way to your earliest data, you can indeed use the
training_period
parameter. By setting a large
training_period
(e.g., covering the full historical range you want to backfill), Elementary will collect metrics for that entire period, as long as you have a
timestamp_column
configured. This is the recommended approach for timestamp-based tests, as the
training_period
defines how far back Elementary looks to build its training set for anomaly detection. For example:
Copy code
yaml models:   - name: your_model     tests:       - elementary.volume_anomalies:           timestamp_column: your_timestamp           training_period:             period: day             count: <number_of_days_to_cover_all_history>
[training_period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period) Process: 1. Set
training_period
to cover your full historical range. 2. Run the test to collect metrics for that period. 3. After the initial backfill, you can reduce
training_period
to a more typical value for ongoing monitoring. Important: - For regular tables/views, the full
training_period
is recalculated on each run. - For incremental models/sources, the full
training_period
is calculated on the first run and on full refresh; subsequent runs only calculate the
detection_period
unless you do a full refresh. - If you change
training_period
after running tests, you need to do a full refresh of the metrics to update the collected data ([Anomaly troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)). So, yes, you can backfill by running the test once with a large
training_period
, then set it back to a normal range for future runs. If you change the period later, remember to do a full refresh to update the metrics accordingly.
c
@kapa.ai If I do a full refresh of the table, it will effect how elementary uses training_period? It will recalculate the whole table for the training_period and drop any data that is in the recorded metrics prior to the training_period?
k
A message from kapa.ai
Yes, if you do a full refresh of the relevant Elementary model (such as
data_monitoring_metrics
), Elementary will recalculate metrics for the entire
training_period
you have configured. Any data in the recorded metrics that is prior to the new
training_period
will be dropped—only data within the current
training_period
(plus detection period, if relevant) will be retained after the refresh. This behavior is explicitly described in the documentation: > If you change var
training_period
in your
dbt_project.yml
, you will need to run a full refresh to the metrics collected. This will make the next tests collect data for the new
training_period
timeframe. > 1. Change var
training_period
in your
dbt_project.yml
. > 2. Full refresh of the model ‘data_monitoring_metrics’ by running
dbt run --select data_monitoring_metrics --full-refresh
. > 3. Running the elementary tests again. > > [Source: Troubleshooting](https://docs.elementary-data.com/oss/general/troubleshooting) Additionally, for incremental models and sources: > The values of the full
training_period
period is calculated on the first test run, and on full refresh. The following test runs will only calculate the values of the
detection_period
period. > > [Source: training_period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period) So, after a full refresh, only the data within the current
training_period
(and detection period) will be present in the metrics, and older data will be removed.
c
@kapa.ai Can you show me an example of how to configure training_period on dbt_project.yml instead of individual test yamls?
k
A message from kapa.ai
Based on the knowledge sources, you can configure global variables for Elementary in the
vars
section of your
dbt_project.yml
. However, it is important to note that while some parameters can be set globally, there are known limitations and bugs regarding which parameters are actually respected at the global level. Specifically,
training_period
is documented as a variable that can be set globally, but in practice, it may only be supported at the test level due to current implementation limitations and bugs. Here is an example of how you would configure
training_period
in your
dbt_project.yml
under the
vars
section:
Copy code
yaml vars:   training_period:     period: day     count: 90
However, according to recent support discussions,
training_period
and
detection_period
are only reliably supported on a test level at this time, and setting them globally in
dbt_project.yml
may not have the intended effect. This is acknowledged as a bug and is being worked on by the Elementary team. For now, you should set
training_period
directly in your test configuration for consistent results:
Copy code
yaml models:   - name: your_model     tests:       - elementary.volume_anomalies:           training_period:             period: day             count: 90
[See Slack discussion for details](https://slack.com/archives/C02CTC89LAX/p1706795086121929) Summary: You can try to set
training_period
globally in
dbt_project.yml
as shown above, but due to current limitations, it is recommended to set it at the test level for now. If you want to track progress on this feature, you may follow updates from the Elementary team in their community channels.
c
@kapa.ai Is
force_metrics_backfill
and old configuration variable?
k
A message from kapa.ai
No,
force_metrics_backfill
is not an old configuration variable. It is a current variable that you can use when running dbt tests with Elementary to force a full refresh of cached metrics. This is especially useful for incremental models or sources, where metrics are cached and not always recalculated unless explicitly requested. By running your test with
--vars '{"force_metrics_backfill": true}'
, you ensure that all metrics are recalculated, bypassing any caching mechanisms. This variable is not deprecated and is still recommended for situations where you need to refresh all historical metrics or after significant data changes [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1705323587448999) [another example](https://slack.com/archives/C02CTC89LAX/p1698154501770259).
c
@kapa.ai Does
force_metrics_backfill
apply to the training period, or will it ignore the training period and go all the way back to the earliest data?
k
A message from kapa.ai
Based on the available knowledge sources, the
force_metrics_backfill
variable forces Elementary to recalculate and refresh all cached metrics for the relevant period. However, it does not override the configured
training_period
. Instead, it ensures that metrics are recalculated for the period defined by your test configuration (such as
training_period
and
detection_period
). It does not automatically go all the way back to the earliest data unless your
training_period
is set to cover that entire range. In summary: -
force_metrics_backfill
applies to the period defined by your configuration (e.g.,
training_period
), not to the entire history of your data unless you explicitly set the
training_period
to cover all history [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1708533386888479) [troubleshooting guide](https://docs.elementary-data.com/oss/general/troubleshooting). If you want to backfill metrics for all historical data, you must set your
training_period
accordingly and then use
force_metrics_backfill
to ensure all those metrics are recalculated.
c
@kapa.ai Is there a way to silence warnings and errors only for elementary tests so I don't need to duplicate this config for each test?
Copy code
config:
            error_if: "= -1"
            warn_if: "= -1"
k
A message from kapa.ai
Based on the knowledge sources, there is currently no supported way to globally silence warnings and errors only for Elementary tests without duplicating the config for each test. Several users have asked about setting severity or silencing behavior for all Elementary tests at once, and the consensus from Elementary support is that this is not natively supported at the project or package level at this time. You must set the configuration (such as
severity: warn
or custom `error_if`/`warn_if` conditions) individually for each test if you want to override the default behavior [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1690908170163389) [another example](https://slack.com/archives/C02CTC89LAX/p1706126443387969). There have been suggestions and feature requests to allow for global configuration of severity or silencing for Elementary tests, but as of now, this functionality does not exist. If this is important for your workflow, you may consider opening a feature request with the Elementary team. Summary: You currently need to duplicate the config for each Elementary test to silence warnings and errors, as there is no global configuration available for this purpose.
c
@kapa.ai You're sure I can't configure it like this in my dbt_project?
Copy code
models:
  +copy_grants: true
  dbt_artifacts:
    +schema: dbt_artifacts

  elementary:
    +schema: dbt_elementary
    +tags: ["elementary"]
      volume_anomalies:
        error_if: "= -1"  # Never error
        warn_if: "= -1"   # Never warn
k
A message from kapa.ai
Based on the knowledge sources, you cannot configure Elementary test silencing in your
dbt_project.yml
in the way you described. While you can set some configurations for models and tests at the project level, the specific approach of nesting test-specific configs (like
error_if
and
warn_if
for
volume_anomalies
) under the
elementary
section in
dbt_project.yml
is not supported by dbt or Elementary. Elementary tests do not support global silencing (e.g., setting
error_if: "= -1"
and
warn_if: "= -1"
) for all tests via a single configuration block in
dbt_project.yml
. You must set these configurations individually for each test. This limitation has been confirmed in multiple community discussions, where users have asked for a way to apply such settings globally and have been told it is not currently possible [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1690908170163389) [another example](https://slack.com/archives/C02CTC89LAX/p1706126443387969). If you want to silence warnings and errors for Elementary tests, you will need to duplicate the config for each test. Summary: No, you cannot configure Elementary test silencing globally in
dbt_project.yml
as shown in your example. Each test must be configured individually.