This message was deleted.

Hey! Yes, we have the caching graph adapter for this. https://hamilton.dagworks.io/en/latest/reference/graph-adapters/CachingGraphAdapter/ Should be something pretty easy to get started with. Note this capability is something we’re going to be adding to over time (more intelligent caching, etc…), but it should provide the basic capabilities you need to do this 🙂

Thanks for your superquick reply! Although the project has a detailed doc, the concept is still a little bit abstract for me, so I asked some stupid questions. Sorry for that~

Not stupid at all! In fact, I think this is something that we really want to be more first-class (and have a few plans in the works to do so). Also we definitely need an FAQ to centralize stuff — lots of documentation but its fairly spread out 🙂

Good! Compared to a lot of workflow I pretty much like yours. It is elegant and clever. I started to use this one to build my scientific workflow and I will contribute to it if I have generalized methods. Thanks!

That’s music to our ears! We love contributions — will happily welcome it 🙂 Even general examples are great (E.G. workflow management using slurm). And feel free to ask qs of course, its great feedback for us and odds are that things you’re attempting are aligned with our roadmap in some way or another. Can help us prioritize.

🙌 Sure, definitely!