Hi everyone! I get more and more like Hamilton and really enjoy using it in a functional way! I wonder if I can use

@resolve

expand a (template) function to

n

(not fixed, determined when

CONFIG_AVAILABLE

) functions with different inputs, and collect them in the next function:

from hamilton.function_modifiers import resolve, ResolveAt, parameterize_sources
from hamilton.function_modifiers import source, value

@resolve(
        when=ResolveAt.CONFIG_AVAILABLE,
        decorate_with= lambda names: parameterize_sources(
            **{
                f'parallel_{name}': dict(name=name) for name in names
             }
        )
)
def serial(name: str) -> str:
    # expand serial func to n functions
    # n is not fixed
    return name

@resolve(
        when=ResolveAt.CONFIG_AVAILABLE,
        decorate_with= lambda names: parameterize_sources(
            **{
                f'collect': {f"parallel_{name}": value(name) for name in names}
             }
        )
)
def collect(all_name: tuple[str]) -> str:
    # expected input node: parallel_a, parallel_b, parallel_c
    # each value of input is calculated by serial function
    #     serail is like a template
    return ' '.join(all_name)

# above equal to:
def parallel_a:...
def parallel_b:...

def collect(parallel_a, parallel_b, ...):...
# but the number of inputs of collect is not fixed, determined when config_available

I think it could be achieved with Parallelable/Collect, but this "static & compile" style looks fancy, and it maybe more friendly to graph/UI?

in case you want to know why: A molecule construct with different segments, and each segments need to be processed(and calculated) in advanced. So a general workflow dont know how many segments(inputs) would be taken before "CONFIG_AVAILABLE". I hope I can define a template, and specialize it when it start.

Glad you’re happy! So yeah, if I understand correctly, you’ll want

inject

— https://hamilton.dagworks.io/en/latest/reference/decorators/inject/

Thanks! It looks like what I want! What style is better? I found a suggestion in the page of

resolve

:

However, since the code goes against one of Hamilton's primary tenets ( that all code is highly readable), we require that you enable power_user_mode.

Only because this style is not readable? I also think this method is hard to parallelize by executor?

Yeah, so, I just put that there to be careful. As in the code can become absolute slop if you’re not careful about it, but it can unlock quite a few capabilities. That’s why I usually suggest folks build their own wrapper over it (e.g. a decorator that calls it), which enables them to make it readable in a custom way.

Thx!! i will build two prototypes to understand those two styles

Awesome! And if you like the pattern/think others can use it you can contribute it back. We can stick it in experimental for now and see if others use it.

ofc! I am still learning the code and understanding the philosophy. I am moving my naive code to a rational-designed platform as my first research article. I can find out which code is general and useful, then I can start to contribute😄

Yeah! I think this is a common pattern (you’re not the first person to ask), so if you come up with something you like definitely send it our way