<@U088WCP3B4P> What time column does the -`-days-b...
# ask-elementary-ai
o
@kapa.ai What time column does the -
-days-back
command use?
k
A message from kapa.ai
The
--days-back
parameter in Elementary Data uses the
timestamp_column
that you configure for your models to determine the time range for collecting metrics and analyzing anomalies. When you define a
timestamp_column
in your model configuration, Elementary uses this column to divide the history of your table into time buckets (typically days). These past time buckets are then used as a training set for anomaly detection. For example, if you set
--days-back 30
and have configured a
timestamp_column
, Elementary will look back 30 days from the current date using that timestamp column to gather metrics and create a training set for anomaly detection. If you don't define a
timestamp_column
for a model, Elementary will instead save the entire row count of the table each time the test runs, without splitting the history into time buckets. In this case, the
--days-back
parameter still controls how many days of historical test runs to use as training data. You can configure the
timestamp_column
in your model YAML like this:
Copy code
yaml models:   - name: your_model_name     config:       elementary:         timestamp_column: "your_date_column"
And then use the
--days-back
parameter when running Elementary commands:
Copy code
edr report --days-back 30
Note that for certain seasonality settings, the
days_back
value might be multiplied. For example, with
seasonality: day_of_week
, your
days_back
value would be multiplied by 7 to ensure enough data for each day of the week [Link to Slack discussion](https://slack.com/archives/C02CTC89LAX/p1696533226544519). By default, Elementary uses 14 days as the training period if you don't specify a
days_back
value [Link to Slack discussion](https://slack.com/archives/C02CTC89LAX/p1650989426047689).
o
@kapa.ai Can you link me the sources that mention these?
k
A message from kapa.ai
I'd be happy to provide the sources that mention the information about the
--days-back
parameter and how it relates to the
timestamp_column
in Elementary Data. Here are the relevant sources from the knowledge base: 1. [Training Period Documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period) - This explains how the training period works, which is related to the
days_back
parameter. It mentions that the default is 14 days and explains how it works with timestamp columns. 2. [Slack Discussion on Seasonality](https://slack.com/archives/C02CTC89LAX/p1696533226544519) - This thread explains how seasonality settings multiply the
days_back
value (e.g., with
day_of_week
seasonality, it's multiplied by 7). 3. [Slack Discussion on Training Period](https://slack.com/archives/C02CTC89LAX/p1690997004224019) - This explains the default lookback window of 14 days and how it affects test behavior. 4. [Slack Discussion on Timestamp Column](https://slack.com/archives/C02CTC89LAX/p1690901726388079) - This thread discusses how
days_back
works with and without a timestamp column. 5. [GitHub Issue on Timestamp Filtering](https://github.com/elementary-data/elementary/issues/1158) - This issue discusses how the
days_back
parameter relates to filtering by the
timestamp_column
. These sources collectively explain how the
--days-back
parameter works with the
timestamp_column
in Elementary Data's anomaly detection system.
o
@kapa.ai Can you filter edr monitor command to capture only the last invocation?
k
A message from kapa.ai
Yes, you can filter the
edr monitor
command to capture only the last invocation by using the
--select
flag with the
last_invocation
parameter. This allows you to send alerts only for the most recent dbt run. Based on the knowledge sources, here's how you can do it:
Copy code
edr monitor --project-dir=<dbt project dir> --select last_invocation
Additionally, if you want to only send alerts for tests that failed in the most recent run, you can use:
Copy code
edr monitor --project-dir=<dbt project dir> --select result:error
Note that in older versions,
-s
was used instead of
--select
, but this has been deprecated as mentioned in a Slack discussion where a user received the warning: "'-s' is deprecated and won't be supported in the near future" [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1674118291945399). It's also worth noting that if you're experiencing issues with receiving alerts for previous failed runs even when the latest run was successful, running
edr monitor
after every dbt command is recommended to get the alerts as soon as possible and avoid false alarms [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1712245885735629). If you're looking to filter the report rather than alerts, the
edr report
command also supports the
--select last_invocation
parameter [Generate observability report](https://docs.elementary-data.com/oss/guides/generate-report-ui). I hope this helps with your Elementary Data monitoring needs! End File# kapa-ai/chatgpt-retrieval-plugin # data/elementary/elementary_data_test_results_table.md Human: What is the elementary_test_results table? AI: # Elementary Test Results Table The
elementary_test_results
table is a core component of Elementary Data's observability platform. It stores the results of all tests executed through Elementary, including both standard dbt tests and Elementary's anomaly detection tests. ## Purpose and Structure The
elementary_test_results
table serves as a central repository for test results data, enabling: 1. Historical tracking of test executions 2. Visualization in the Elementary UI 3. Alert generation for failed tests 4. Analysis of test performance over time The table contains detailed information about each test execution, including: - Test metadata (name, type, model) - Execution details (timestamp, invocation ID) - Test status (pass/fail/error) - Failure details and error messages - Test-specific metrics and thresholds ## Location and Access The
elementary_test_results
table is created in your data warehouse when you install and run the Elementary dbt package. It's typically located in the schema specified in your Elementary configuration (default is
<your_schema>_elementary
). You can query this table directly in your data warehouse to analyze test results or build custom monitoring solutions on top of Elementary's data. ## Integration with Elementary Features The
elementary_test_results
table powers several key Elementary features: - Data Observability Dashboard: Visualizes test results and trends - Alerting System: Identifies test failures for notification - Anomaly Detection: Stores metrics and anomaly detection results - Test History: Tracks test performance over time ## Example Query To see recent test failures, you might run:
Copy code
sql SELECT    id,   test_unique_id,   test_name,   status,   executed_at,   error_message FROM    your_schema_elementary.elementary_test_results WHERE    status = 'fail' ORDER BY    executed_at DESC LIMIT 10;
For more information on how to leverage the data in this table, refer to Elementary's documentation on [querying test results](https://docs.elementary-data.com/guides/data-tests/query-test-results).