I have a question when doing backfilling data.
For example: My table is incremental. I set the detection period to 1 and run the dbt models (today is 7/2). It goes back to the last 98 days (14*7) (3/27) and make prediction on the most recent date (7/2). However, when I rerun the test, it adds that most recent date to the training set size (98 -> 99) and predicts the same day (7/2). So is it synonymous to using new data to retrain and predict for that day?
I did this because I am thinking the problem when doing backfilling, e.g we lost data on a specific day, we want to twist the detection period to 10 days and rerun the dbt models, it will also have the same problem as above right, right? (It also used data trained for those 10 days (including days with 0 records) for a new training set right?
I am thinking to the usage of by setting
force_metrics_backfill = true
but it seems that it creates a new training set but still uses old data for training. (the training size now for the most recent date is around 190 instead of 98)