Hi, can I ask what

later calls will take precedence

refers to in the docstring of with_modules? I was wondering if it’s possible to have 2 modules,

module_a

and

module_b

, that both implement a

common_node

node, and calling

builder.with_module(module_a)

and then

builder.with_module(module_b)

would use the

common_node

from

module_b

to build the DAG - but this doesn’t seem to be the case.

Hey — so I think that this docstring is wrong. In that it uses all of them together — it will find two definitions for

common_node

, and error out. Just to ensure — what happens when you try? Otherwise “overriding” functions is something we’re thinking about but I don’t think we have the capability yet. The common approach is to switch between a/b depending on a config, and parse out common ones.

Yup, it errors out saying there’s a duplicate node! Ok, I was hoping to avoid writing

__a

and

__b

with

config

, but sounds like that’s the best bet to implement this for now?

you don’t need the

__

suffix if they’re in different files

Ah that’s great, thanks @Stefan Krawczyk!

yeah the suffix is only required because you can’t have two python functions with the same name in a module…

only required because you can’t have two python functions with the same name in a module

That makes a lot of sense!

this also means that if you run the modules individually, you’ll need to provide config values

Noted, a small inconvenience

for me, it’s more around how would someone debug / understand which one was being used?

how about adding a flag to

with_modules(*module, override=False)

that has to be explicitly turned on?

So I like having this as a feature (we’d discussed this in the past), but I think that it’s a conscious decision

allow_verride=True

hmm, I think with the

with_modules

would be more contextually relevant personally…

Yeah, as long as it’s clear that any call to

allow_overrides=True

allows overrides globally (which is iffy). That said, very few people use multiple

with_modules

calls, so I’m not super concerned, as long as we document.

Oh I see.

any call to

allow_overrides=True

allows overrides globally

Just to check my understanding, this means in a sequence of multiple

with_module()

calls, if any one call has

allow_overrides=True

, the overriding behavior will be applied to the entire sequence of modules added to the builder?

Yeah — that’s the easier implementation + easier to understand API. But that’s why I suggested globally — it makes it a bit clearer (although not as contextually readable).

Thanks for clarifying, I don’t have a strong preference but the global call seems cleaner 🙂

@Yijun Tang do you mind creating an issue for this feature request?

sure, will do!

@Yijun Tang hey! @Jernj Frank actually took on this feature (🙌 ), and I created an RC version if you’re interested in trying!

sf-hamilton==1.77.0rc0

has it. You can see it here: https://github.com/jernejfrank/hamilton/blob/feature/override_nodes_at_module_import/examples/module_overrides/run.py — should be simple as just adding the

.allow_module_overrides()

call on the builder

Awesome, thanks so much! 🙌🙏