Question 2: What a good way to represent database access that is visible and clean in HamiltonUI?
The initial part of my current data pipeline is the lookup and assembly of the initial configuration. It requires database access to resolve string references into real Python objects and some minor logic. I can do this outside the execute but I also tried feeding execute a plain config (YAML) file and a duckdb connection to have it resolve into a typed instance of a class. I'd like to have as much of the code inside the graph but I'm wondering if this is a good pattern. It seems like it given your examples but I'm also wondering if I'm missing something with materializers and adapters.
The graph looks a bit odd since a lot of nodes access the db provided as input. Still a good path?
s
Stefan Krawczyk
04/24/2025, 5:37 PM
Yeah we need to add a flag to make the graph look less messy in the case of things like DB connection objects.
In terms of materializers, they're just a way to hide/centralize logic. So depends what you're optimizing for.