Hi, I just try Hamilton today and have a question....
# hamilton-helpc
Cuong Truong Huy
08/01/2024, 6:00 PMHi, I just try Hamilton today and have a question. Is there any way to reuse a function (as node) in N times?
The idea is to create a for-loop but in node-level.
👀 1
s
Stefan Krawczyk
08/01/2024, 6:05 PMYep - have a look at https://www.tryhamilton.dev/tutorial-extras/parameterization and see whether it meets your needs.
Stefan Krawczyk
08/01/2024, 6:08 PMMore documentation here - https://hamilton.dagworks.io/en/latest/reference/decorators/parameterize/
Stefan Krawczyk
08/01/2024, 6:08 PMIf you need it to be dynamic — then you can connect it with resolve.
y
Yijun Tang
08/01/2024, 6:20 PMIs it possible to dynamically generate the
parameterizes
Stefan Krawczyk
08/01/2024, 6:28 PMHow dynamic? Something that you pass into the driver? Or just a list you define in the module? Or?
Stefan Krawczyk
08/01/2024, 6:34 PMShort, answer is yes it can be. Can point to code. (See resolve link above for example).
y
Yijun Tang
08/01/2024, 7:18 PMSomething passed into the driver. Let’s say in the main script, we load some config parameters from yaml, then construct a list of parameters from the yaml config, then pass the list to the driver
s
Stefan Krawczyk
08/01/2024, 7:24 PMyep, so
@resolve👀 1
Stefan Krawczyk
08/01/2024, 7:28 PMyou’d want something like:
Copy code
from hamilton.function_modifiers import resolve_with_config, parameterize, source, value
@resolve_with_config(
decorate_with=lambda config_key1: parameterize(
**{f"node_name_{x}": {"s1": ..., "s2": ...} for x in config_key1}
)
)
def summation(s1: pd.Series, s2: pd.Series) -> pd.Series:
return s1 + s2 Copy code
from hamilton import settings
_config = {settings.ENABLE_POWER_USER_MODE:True}
# this should match the name in the lambda
_config["config_key1"] = ["sub_metric_1", "sub_metric_2"]
# Then in the driver building pass in the configuration:
.with_config(_config)Stefan Krawczyk
08/01/2024, 7:29 PMnote: you can break the inside of the decorator out into functions if it gets complex/harder to read.
y
Yijun Tang
08/01/2024, 8:24 PMwow that’s cool 👍
🙂 1