New message text here

@kapa.ai is the where clause to filter only last month's data making any difference at all if I keep the default config for detection & training periods?

If you use a where clause to filter only last month's data, but keep the default configuration for detection and training periods (which are 2 days and 14 days, respectively), the where clause will limit the data Elementary processes to just that subset (i.e., last month's data). This means that both the training and detection periods will be calculated only within the filtered data. However, if your where clause restricts the data to a single month, and the default training period is 14 days, Elementary will only have up to 14 days of data (from that month) to use for training, and 2 days for detection. If your table contains more than a month of data but the where clause restricts it, Elementary will not "see" the older data outside of that month. So, the where clause does make a difference: it restricts the data available for both training and detection periods to only the rows that match the filter. If you want Elementary to use a longer history for anomaly detection, you should either remove or adjust the where clause accordingly [all_columns_anomalies test configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/all-columns-anomalies). In summary: Yes, the where clause is effective and will limit the data Elementary uses for both training and detection periods to only the filtered subset. If you keep the default periods, they will be calculated within the filtered data only.

New message text here

You cannot specify the

column_anomalies

monitors at the model level inside the

config.elementary

section or in the

dbt_project.yml

file. According to the documentation and community discussions, the

column_anomalies

parameter must be set at the test level, not at the model or global config level. This means you need to define which monitors to run for each test, either in the model's YAML file under the specific test or directly on the column test configuration. Here’s an example of the correct usage at the test level:

yaml models:   - name: your_model     columns:       - name: your_column         tests:           - elementary.column_anomalies:               column_anomalies:                 - null_count                 - average

Attempts to set

column_anomalies

at the model level or in

dbt_project.yml

will not work and are not supported by Elementary or dbt’s configuration system [docs.elementary-data.com column_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/column-anomalies) [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1658915396893639). If you want to apply the same monitors to many columns, you must repeat the configuration for each column or use YAML anchors for reuse, but not at the model or project config level.

No, you cannot set the

column_anomalies

parameter in the

vars

section of your

dbt_project.yml

file. According to the knowledge sources,

column_anomalies

is a test-level configuration and must be defined directly within the test block for each column or model. Attempts to set it at the project level (such as in

vars

or under

config.elementary

) will not work, as confirmed by both documentation and community support responses [docs.elementary-data.com column_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/column-anomalies) [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1658915396893639). Other configuration options (like some global thresholds or variables) may be set in

vars

, but not the list of monitors for

column_anomalies

.

New message text here

Yes, according to the current behavior of Elementary, the detection period is included in the training period when calculating averages and standard deviations for anomaly detection. This means that the data points being checked for anomalies (the detection period) are also part of the set used to determine what is "normal" (the training period). This has been confirmed in both community discussions and GitHub issues, and is recognized as a point of confusion and potential improvement in the product. There is currently no built-in configuration to exclude the detection period from the training period, though this has been requested by users and discussed by the maintainers [GitHub Issue #1491](https://github.com/elementary-data/elementary/issues/1491) [GitHub Issue #1560](https://github.com/elementary-data/elementary/issues/1560) [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1712927784827779). If you need to fully separate training and detection periods, this is not currently supported out-of-the-box.