https://github.com/stitchfix/hamilton logo
Join Slack
Powered by
Are the `config` options available as metadata any...
# hamilton-help
a
Are the 
config
options available as metadata anywhere on the function? Specifically, I'd like to check: For each function in a module, get the list of 
@config.when_in(...)
values as well as the keys.
e
So, these aren’t attached directly to the function, although it is certainly an interesting request. To be clear, you’re looking for some metadata that says “Here are the config values that this function can use”, right? It’s a little tricky for a few reasons — the keys are known in most cases, but the values are not always known. Taking a step back though, how would you plan to use it?
a
Yep, I noticed that right now the decorators just wrap those fields in a callback. I'm trying to build a testing function that takes in a module and loops through all possible configs -- hence requiring the tag values
e
Yeah, makes sense. It’s a good question. Some details for a deep-dive (that you might appreciate) 1. Decorators set an attribute on the function — there are a few: https://github.com/DAGWorks-Inc/hamilton/blob/5585c24c7b3813fcfadf180c8c24771e43781e15/hamilton/function_modifiers/base.py#L139 2. This attribute corresponds to a decorator step (there’s internal precedence) 3. There are a few. For instance, with 
@config.when(…)
, the lifecycle step is 
resolve
meaning that the decorated function is 
resolve
4. This can give you the optional/required config Here’s a quick illustration:
Copy code
In [1]: from hamilton.function_modifiers import config

In [2]: @config.when(foo="bar")
   ...: def a() -> int:
   ...:     return 1

In [4]: a.resolve
Out[4]: [<hamilton.function_modifiers.configuration.config at 0x14db715d0>]

In [5]: a.resolve[0]
Out[5]: <hamilton.function_modifiers.configuration.config at 0x14db715d0>

In [8]: a.resolve[0].required_config()
Out[8]: []

In [9]: a.resolve[0].optional_config()
Out[9]: {'foo': None}
This is all fairly internal APIs, however, and doesn’t have the data you want (the values)
Also, not all decorators know in advance the required configuration items as well, but in simple cases we do. So, currently, we don’t really store it. That said, I’d suggest one of two ideas: 1. List out a set of “supported” configs to test — this is nice as it is also self-documenting. Won’t work if you truly want to support an explosion of capabilities. 2. Create a wrapper decorator on top of configs that also attaches metadata to the function. (2) would look something like this (non-tested):
Copy code
@my_config_when(foo="bar")
def a() -> int:
    return 1

assert a.__config == {"foo" : "bar"}
Copy code
def my_config_when(**config_vars):
    def wrapper(fn):
        setattr(fn, "__config", config_vars)
        return config.when(**config_vars)(fn)
    return wrapper
a
Ah, I like the wrapper function idea. btw, I also saw this todo in the code -- maybe functools.wraps is what you're looking for?
e
Great! Yeah I think if you really need the combinatorial explosion that’s a nice approach. Also reduces the configuration/design space, which is the only thing that makes it possible. Heh that TODO is very old — it’s a bit sloppy as it changes the name to communicate with a downstream tool that picks it up. I think there’s a better way to do it but I haven’t dug in. Generally a fan of functools.wraps, but I think we’re doing weird enough stuff that it wouldn’t solve this problem? Also quite possible I’m over-complicating it (and this name-changing code is not necessary anyway)
Open in Slack
PreviousNext
Powered by