I think base memory requirements for NFCORE_MAG:ma...
# mag
j
I think base memory requirements for NFCORE_MAG🔍ALE should be higher than 6Gb. In my 240 sample single assemblies, only 5 used <6. Mean requirement was 14Gb±4.5 with max 24Gb ; for coassembly (in my case 4-sample groups, I know that can vary a lot) the mean was 26±7 with range 9.4-42 Gb I know it's a pretty fast process so retries aren't that annoying or costly but still I think slightly increasing it wouldn't hurt, and there could be a base boost when coassembly is performed too. Here's my suggestion for conf/base.config :
Copy code
withName: 'NFCORE_MAG:MAG:ALE' {
        cpus = 1
        memory = { params.coassemble_group ? 30.GB * task.attempt : 18.GB * task.attempt }
    }
or something like this, which would minimize the repeats. Obviously that's based on a single project but since it has 240 samples with extremely variable sequencing depth (good ol' human DNA in saliva samples...) I think it's a good place to start, because nearly 100% of my jobs had retries (and half had 2 retries) for single assembly. Edit: I guess mag stands for magnifier now 🔍 🙃
d
I came to the same conclusion on the tests I'm running now. I plan to analize all processes when I finish this run and check all the ones that need tweaking. ALE also had extra problems: it was not running in metagenome mode, and was producing massive files. A fix for that is in dev already: github.com/nf-core/mag/pull/1063
🚀 1
If you can share with us your nextflow report and trace once you finish, that would be extremely useful!
j
will do !