Hi 🙂 I am building a pipeline where my input datasets of the same type (scRNA-seq) can differ quite a bit in size. I would like to adjust the resource usage of the processes where these datasets are used as input, depending on the size of the dataset. My first thought was to add a column to the samplesheet that includes a scaling value for each dataset, which would indicate the resource in-/decrease for the affected modules. Is this a good solution or are there already other, established setups for this use-case?

I'm interested in the answer too ! I have a similar issue, and I have "solved" it by increasing the memory parameter at each retry :

withName: 'prediction' {
        memory = { 16.GB * Math.pow(2, task.attempt - 1) }
    }

Sure, it's not the most optimal way, but it's the best I've found so far...

There are a few modules that take the size of an input to set resources like:

memory { 6.B * fasta.size() }