Hello everyone
I want to use the same DAG for both training/updating the model and for inference.
During inference, the DAG should load all learned parameters and artifacts from the database, do prediction and output it.
During training, the DAG should also load the artifacts if they exist, and do training or updating it and then save it to the database.
What's the best practice to architect the DAG in order to use it for these two purposes. Thanks a lot.
Note: I know that the two purposes are going share some nodes and will be different in the downstream nodes, but at least I am looking for a clean way to utilize the starting dag because as you know the training and inference pipelines share the same nodes at the beginning and then they branch out