Hi there! Situation: The elementary test was run for some table for some time. After this, the data in the table was dropped and new ones were added, that is, now the test needs to be trained on new data. But the test turns out to be trained on old data. What should I do in such a situation?

Hi @Anna Vasileva, can you try to rerun the test but add

--vars '{"force_metrics_backfill": true}'

at the end of your dbt command?

Oh thanks apache spark I haven't seen anything like this in the doc I don't have that problem right now, but we are analyzing whether we can use this tool for our needs.

oh ok got it 🙂 the metrics are supposed to update every time but in case they don't this is a safety you can use

Do you mean that by default they should be re-expanded for the entire training period (for example, 14 days)?

in the recent version we made some optimizations and now the logic is as follows: 1. the data within the backfill_days (which is the detection period) is being re-populated every time the test runs 2. the data in the training period (days_back) is only being queried if there are gaps (for example, the detection period is 2 days but the test hadn't run for 5 days, then we will query the last 5 days) 3. if the data in the training period has changed we don't have a way of knowing, but you can use

force_metrics_backfill

and then it will always be updated (this is not being done by default as a performance optimization)

Thanks for the detailed explanation!