Yijun Tang
08/13/2024, 1:23 AM@tag(node_type="output", table_name="my_table") . Is there a way to make sure that any node tagged with node_type="output" must also have a table_name , other than creating a decorator by wrapping tag ?
2. How do you recommend checking all node_type="output" nodes have a required output type OutputType ? I’m thinking about using the type class variable on the nodes.
Thank you!Stefan Krawczyk
08/13/2024, 1:37 AMYijun Tang
08/13/2024, 1:38 AMElijah Ben Izzy
08/13/2024, 2:01 AMYijun Tang
08/13/2024, 2:24 AMElijah Ben Izzy
08/13/2024, 2:25 AMStefan Krawczyk
08/13/2024, 3:00 AMYijun Tang
08/13/2024, 1:57 PMStaticValidator looks like exactly what I’m looking for! Would appreciate some simple examples, especially on how to traverse the nodes in a graph: HamiltonGraph , and what attributes are available on a node: HamiltonNode for validation purposes. 🙏Stefan Krawczyk
08/13/2024, 4:10 PMfrom typing import Optional
from hamilton.lifecycle import api
from hamilton.graph_types import HamiltonNode
class MyTagValidator(api.StaticValidator):
def run_to_validate_node(
self, *, node: HamiltonNode, **future_kwargs
) -> tuple[bool, Optional[str]]:
print(node.tags)
if node.tags.get("node_type", "") == "output":
table_name = node.tags.get("table_name")
if not table_name: # None or empty
error_msg = f"Node {node.tags['module']}.{node.name} is an output node, but does not have a table_name tag."
return False, error_msg
return True, None
if __name__ == '__main__':
from hamilton import driver
import good_module
tag_validator = MyTagValidator()
dr = (
driver.Builder()
.with_modules(good_module)
.with_adapters(tag_validator)
.build()
)
print(dr.execute([good_module.foo]))
import bad_module
# this should error
dr = (
driver.Builder()
.with_modules(bad_module)
.with_adapters(tag_validator)
.build()
)
# good_module.py
from hamilton.function_modifiers import tag
@tag(node_type="output", table_name="my_table")
def foo() -> str:
return "Hello, world!"
# bad_module.py
from hamilton.function_modifiers import tag
@tag(node_type="output")
def foo() -> str:
return "Hello, world!"Stefan Krawczyk
08/13/2024, 4:26 PMYijun Tang
08/13/2024, 5:18 PMrun_to_validate_node will be applied to all nodes in a DAG?Stefan Krawczyk
08/13/2024, 5:19 PMrun_to_validate_node will be applied to all nodes in a DAG?
correct. after .build() is called.