Hi everybody, I'm trying to figure out what is the best Hamilton pattern for evaluating multiple different models, such as model_A and model_B, using the same shared evaluate function. Is it better to run the pipeline multiple times with different configurations for each model, or to define one large pipeline that computes all model evaluations in parallel? For example:

# We have two different model functions:
def model_A() -> TrainedModelA: ...
def model_B() -> TrainedModelB: ...

# We have ONE shared evaluation function:
def evaluate(model: AnyTrainedModel) -> float: ...

# How do we use Hamilton to get the results of `evaluate(model_A)` and `evaluate(model_B)`?

When talking about configuration, I mean

config.when

and then providing the values in for loop to

Builder

. For the second cast, to avoid code duplication, using something like

@parametrize

. I've found model training docs but unfortunately neither of them is describing the pattern I have in mind. Just to be clear, the goal is to: a) train multiple models "at once" b) find out the best of them - usually by looking and comparing the evaluation results in something like MLFlow Thanks!

yeah, no one best way.

Wow, thanks for prompt answer! Haven't noticed

subdag

abstraction / the whole modular perspective looks very insightful! That looks like a nice building blocks as the size/complexity of the project rises. For the record, I've settled for

@config.when

with manual gluing of drivers for now as the setup is fairly simple but

subdag

looks like a way to go in the future. Thanks a lot 😊