Yes, you can use

elementary.freshness_anomalies

to check for significant gaps at the end of your table, which would indicate ingestion issues preventing new entries from being copied from the source table. The

elementary.freshness_anomalies

test monitors the freshness of your table over time by analyzing the expected time between data updates. It works by splitting your data into time buckets (daily by default) based on a specified

timestamp_column

, and then comparing the freshness of recent buckets to historical patterns. If there is an unexpected delay (i.e., a significant gap since the last update), the test will fail and alert you to the anomaly. This is exactly the scenario you described—detecting when new data has not been ingested as expected within a 24-hour period. Here’s a sample configuration for your use case:

yaml models:   - name: my_table     tests:       - elementary.freshness_anomalies:           timestamp_column: <your_timestamp_column>  # Replace with the column indicating data arrival           time_bucket:             period: day             count: 1

Make sure to set the

timestamp_column

to the field that records when each row was added to

my_table

. The default configuration is already set to daily buckets, which matches your 24-hour ingestion cycle. The test will alert you if there is a significant delay compared to the learned update frequency of the table [freshness_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/freshness-anomalies). If you want the test to be more sensitive (e.g., to alert on a single missed day), you can adjust the

anomaly_sensitivity

parameter to a lower value, making the test more likely to flag delays as anomalies [slack discussion](https://slack.com/archives/C02CTC89LAX/p1703157711512959). Note: The test requires some historical data to learn the normal update pattern. If your table is new or has irregular updates, you may need to adjust the training period or sensitivity settings for optimal results. For more advanced scenarios (e.g., if you want to check the freshness of the actual data, not just the update time), you might also consider the

event_freshness_anomalies

test, which compares event timestamps to their load times [event_freshness_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/event-freshness-anomalies). In summary,

elementary.freshness_anomalies

is well-suited for your use case of detecting ingestion gaps in a table updated every 24 hours.