https://github.com/stitchfix/hamilton logo
Join Slack
Powered by
Is it acceptable to start an `asyncio.run()` loop ...
# hamilton-help
n
Is it acceptable to start an 
asyncio.run()
loop within a Hamilton node to parallelize some work in that node? It seems to work, but I'm curious if there are gotchas.
I do see the Async driver BTW, but I really want to have caching. I did see https://github.com/apache/hamilton/issues/1263 and that looks to be in progress still. In my use case I am comfortable with the graph being executed synchronously but just want to be able to do some async work in particular nodes. Conceptually my requirements don't look too dissimilar to the collect/gather pattern here: https://hamilton.staged.apache.org/concepts/parallel-task/ But it felt like it might be simpler to just localize the async to my node.
(and I see I can't get a run view graph if using that which I would really like to keep)
IE, will this node cause me trouble:
Copy code
def pdf_image_captions(output_language_config: str, caption_prompt_config: PromptConfig, pdf_pages: list[Page]) -> list[ImageCaption]:
    async def generate_captions():
        captions = []
        for page in pdf_pages:
            for image in page.images:
                captions.append(get_image_caption(caption_prompt_config, page, image, output_language_config))
        return await asyncio.gather(*captions)

    return asyncio.run(generate_captions())
I realize the above could/should be in its own event loop but question still stands.
s
It will only parallelize if it's I/O bound work
But yes, you could in a Hamilton node do something like the above.
I don't see issues. Hamilton functions are pretty well decoupled from each other. So this logic is all inside that node. It should be easy to unit test.
If that processing logic gets complex, you could run async hamilton there too (i.e. hamilton within hamilton)...
n
My use case is a pipeline that does a bunch of processing on a PDF using LLMs, say image captioning for example. One node extracts all the images in the PDF, another goes and captions all the images. The captioning is LLM calls so that can all happen in parallel and that node can then take ~10s total vs 10s x number of images.
šŸ‘ 1
s
@Nic Pottier nice.
e
n
Elijah, I have, and I played with it but I really want to keep run visualizations and felt harder to reason about.
I don't know what would be involved but just having the ability to have a node be async internally (but not externally to the node) would be a really nice feature. I feel like this kind of parallel work within a node has to be a pretty common ask.
IE, having the driver just call 
await
on my node with the execution / resolution being exactly the same as a normal synchronous run.
That would allow for cases like mine without the somewhat scary / not so recommended path of starting my own event loop.
s
if you aren't running in an async context -- you should be able to control the event loop easily, no?
or if you're worried, you can just pass it in as an input parameter and have functions depend on it?
n
@Stefan Krawczyk Yes, I think both of those are possible, but they also both seem more complicated than just having an 
async
node. It also seems slightly riskier in that you then are depending on no other event loops running. Am I underestimating how difficult it would be to have a Hamilton driver that does this? Just feels like a really nice option for those using Hamilton at the command line to be able to speed up individual nodes with minimal complication of the actual node implementation. (and while retaining caching and run introspection via the call graph)
s
@Nic Pottier that's the async driver! But you wanted caching, which isn't there yet, right?
There could be a way to have a regular sync driver check for async somewhere where it wouldn't interfere with caching, but I'd have to look at that code a bit more to see if that's possible.
n
@Stefan Krawczyk My understanding is that the Async driver is a lot more complicated because it attempts to run multiple nodes at once. This would be much simpler conceptually and therefore I don't think would affect caching or run graph generation. Or am I understanding that wrong?
Like an 
AsyncSyncDriver
heh.
(or maybe the default driver could just work that way I suppose?)
s
Nodes can sync or async with async driver.
n
Yep, understood.
s
Running in parallel is a side effect of being able to schedule different branches of dag due to async.
But, you need to run the async driver in some event loop yourself.
n
Right, ya, I understand all that. I was just wondering since I totally understand the complication of having caching work reliably when separate nodes can run in parallel whether a driver that didn't run nodes in parallel but did allow nodes to be async would give us both "native" async support at the node level while retaining caching etc.. It strikes me that this user sort of had the same requirements as me in that respect: https://github.com/apache/hamilton/issues/1263
Anyways, this is all just very non-educated feedback, appreciate you engaging. I have solved my particular issue with having my own event loop but just thought I'd better explain what I would like to see in case that ever rhymed with what someone else later requested. šŸ™‚
(I'm also willing to give it a shot if you think there's any chance of success!)
s
We just need contributions @Nic Pottier šŸ˜‰ Everything is possible with community contributions šŸ™‚
šŸ‘ 1
Otherwise thanks for engaging šŸ™‚ I think we can do a lot more on the async side in general
n
I guess my question is more do you think there's any merit to the idea? I do know how annoying it can be having contributors opening PRs for things you really aren't interested in! If you think there's some merit then I'm willing to give it a go and will open an issue and PR with it. Otherwise happy to stay in my event loop world!
s
What I want -- is a parallel collect async equivalent.
It's a question of what to do when: ā€¢ Ideally we can get caching to work on async... ā€¢ or maybe we rework the drivers (big task) to just have one that can handle both. ā€¢ or we get the sync driver to handle running async functions that will work with caching...
So basically a few things to scope out
Otherwise your product requirement is clear I think -- and could be worth putting in an issue: 1. I want caching 2. I want to run sync and async functions 3. Ideally hamilton can take an event loop, or run inside one Challenge here is that there are multiple paths to resolving this šŸ™‚
n
Ok, great, that's helpful. I'll add an issue just to record that. Agree lots of paths to get there!
Open in Slack
PreviousNext
Powered by