Hello All, I was thinking about feature which estimates ETL completion time, incase of some failure, restart of ETL or full load, so we can inform our stackholders that data will be available in x minutes/hours.

Hi @Amol Jog! thanks for surfacing! do you think about something like

dbt run -s tag:finance

is expected to take 1.2 hours based on historical runs of the models in this DAG? would love to hear more about the use case that you have in mind

@Or Avidov Yes thats what i meant.

That's an interesting use case! as a workaround this can actually be quickly answered by looking at the elementary tables that are created in your warehouse. For example by looking at the duration in the dbt_invocations table or the sum of all model execution times that belong to the same tab

Yes was thinking the same, like creating function where input is model and output is sum of execution time for downstream.