Hi everyone, I’m a student bridging Oceanography and Software Engineering (42 Marseille). I'm building an open-source CLI (

zesthpc

) to run scientific workflows on decentralized infrastructure (Akash Network) to lower cloud costs and DevOps friction. Since

nf-core

is the standard, I want to support Nextflow seamlessly from the start. I would love to ask you two quick questions to avoid building the wrong thing: 1. What is your biggest infrastructure bottleneck right now? 2. What are the typical CPU/RAM requirements for your most frequent runs? The early repo is here if you want to take a look: gitlab.com/zestlab-group/zesthpc/zesthpc Thanks for your time and insights!

We have a wide number of users that uses widely different pipelines on widely different system, so I'm afraid this is not a simple question

Yes, I know, but if I get enough responses, I might be able to spot a trend. And for the people who want to share their needs, they could receive free credits to run their pipelines and give feedback during the CLI beta.

you could check the configs repo: https://github.com/nf-core/configs that would give some info about what people have set up in their infra

Maybe I'm missing something here, but why not write a Nextflow plugin to support Akash as a executor? I don't really see why you need a new CLI tool for this.

Thanks Phil, My long-term goal is to keep ZestHPC engine independent so it can support other tools later (like Snakemake). To leverage Nextflow’s features without reinventing the wheel, I'm going for a hybrid approach: a lightweight Nextflow Executor Plugin that acts as a bridge, delegating the actual Akash deployment to the ZestHPC Python core. This way, nf-core users get a seamless, native experience (

-profile akash

), and the tool stays flexible. Does it make sense for you? I’d love to hear your thoughts on how to efficiently handle data staging/transfers between Nextflow and a decentralized infrastructure like Akash in this kind of setup.

Not reinventing the wheel is always a good idea in my opinion

First day of rain in months tomorrow normally 🥵

Looking forward to that. All the plants in my mom's garden have died I'm afraid