This message was deleted.

Going to be in a meeting for the next hour, but some thoughts: • The initial stitch-fix project was massive-scale (in terms of complexity, not data), so its proven out there (and I can attest that it works) • Architecture seems reasonable to me • You’ll probably want a caching mechanism, althought that’s an implementation detail (and there are a bunch of ways to do this, as well as features we want to integrate) • There’s tooling to allow for inspection of the available nodes — tagging can help you separate final/user-facing nodes from intermediate ones: https://github.com/stitchfix/hamilton/blob/b06cc2e26282fd1dd1bcc36698923a669b65b4f9/decorators.md#L281 @Stefan Krawczyk and/or myself are happy to hop on a call at some point to talk over your use-case if you want!

Great, thanks. I think a scaled-up example could be really useful for the docs. Managing complexity is a main selling point, and from smaller examples you can see how it can help, but there's still a lot of big and little decisions to make when actually applying it to anything complex. Having larger examples could help with that.

Fair comment. Will think about what we can add. Otherwise I believe my AICamp talk https://github.com/skrawcz/talks#aicamp-may-2022 gets at some of what you're asking about. Want to browse that in the mean time to see if it helps you?

I'll check it out. Is there much in the talk that you don't have in the slides?

Just adding more context — but yeah the slides are pretty dense so you might be able to get by without watching the talk 🙂

“Code organization” and “common indices” are probably going to be the most useful. Also as you learn best practices (especially for a complex codebase), we’re happy to add!