Hi @Elon Gliksberg, is this functionality available today? I'm interested in running freshness on a table that is partitioned by a column and would love freshness alerts for each partition. for instance, table1 is partition by a column named "customer" and has a last_update_dt column. I'd love freshness anomaly detection on each customer partition, based on the last_update_dt column. I couldn't seem to find this feature clearly in the docs. https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/anomaly-params or https://docs.elementary-data.com/data-tests/anomaly-detection-tests/freshness-anomalies Thanks!

Hey @Will Skelton! Thanks for reaching out again. As for the original question, in our Cloud platform we have advanced ML based freshness and volume monitors. These use the metadata from the information schema (see attached screenshot) 🙂 Freshness anomalies look at the entire table. You can create a test for each partition with a

where_expression

, but I believe this won't be very efficient. Dimension anomalies count the number of rows per value (customer), is this an interesting monitor for you? Interested to understand, are you looking to monitor data quality issues or a business metric (enough updates from a certain customer)?