<!here> Hi, I was wondering if there's a way to gracefully end an Hamilton Dag run before it completes i.e. In one of the nodes you have a condition that if evaluates to true you can send a kill signal to hamilton?

Good question! We do not have this exactly (no inter-node communication), although “early-exit” is the default with errors. There are, however, quite few ways to solve this type of problem. Curious — what’s the use-case/goals here? There are a bunch of reasons I’ve seen folks want to do something like this.

+1 on more information to expand on what you mean by graceful 🙂

Thank you for the replies. So we use Hamilton to orchestrate inter-dependent functions for calculating some performance metrics for certain real-estate assets. We might have potential errors in nodes say because of input-data quality and we want to end the dag run right there and provide a response to clients with the input-errors

Got it — so this is a really good question. Big Q is how deep inside the DAG are these (potentially errorable) nodes? Are these directly user-facing? Or buried further (E.G. with multiple layers of transformations after that). The challenge is that, currently, Hamilton will error out and discard the results (not give you partial ones). But (4) is a workaround for that… This is also something we’d consider improving (adding alternate execution modes is something we’ve thought about before). So, I think you have a few choices — curious how these work: Some ideas (lmk if you want to see code for these, happy to jot something down): 1. try/except inside the fn for a specific error/have a decorator that returns

null

— then you just capture the null data (works if they’re directly user-facing — propagating null can be really ugly though if they’re not) 2. Run through the whole things with data validators — capture the results of the nodes + the data validator results, then select which ones to include (this is inefficient if there’s further computation, which is my guess, but worth thinking through) 3. Run the DAG in two pieces: (a) the data loading that needs validation, and (b) the metrics transforms. Only request the outputs corresponding to the inputs that validate. A little less explicit/hamiltonic, but could definitely work 4. Create a custom lifecycle adapter to catch, propogate, and discard the results of future errors. (4) is a bit complicated but potentially powerful. The way it would work is: 1. Your nodes that “gracefully fail” throw a specific exception 2. You implement a

NodeExecutionMethod

(https://hamilton.dagworks.io/en/latest/reference/lifecycle-hooks/NodeExecutionMethod/) to catch that exception and return a sentinel value 3. That hook also checks if any inputs are of that value, and also returns that value 4. You pass it to the driver and it replaces execution This way you can short-circuit anything in the future that depends on busted data. Then at the end you get a dict back and handle it. I really like this approach — need to prove it out though. Happy to sketch out some code and if its good we can add it to the std lib!

Yes that works @Elijah Ben Izzy. Thank you very much 🙂

Great! We’re going to put this into default Hamilton — probably expose the errors as well :)

@Jesuseyi Fasuyi we just pushed

1.65.0

that has this adapter if you wanted to try it out. Docs: https://hamilton.dagworks.io/en/latest/reference/lifecycle-hooks/GracefulErrorAdapter/