Pieter Moris
02/28/2024, 10:28 AMsbatch --constraint=mem256/mem128 option. I've added these to the profile's process directive as follows: clusterOptions = { task.mem >= 112.GB ? "--constraint=mem256" : "--constraint=mem128"} (112 because we need to account for ~16 GB buffer on our HPC).
My question is whether I should do the same for the max_memory directive, or if I should just set it to the higher value (i.e. 240 GB). From reading the docs I've understood that this value is used to restrict the resource requests of specific tasks, but I'm not sure if task.mem is based on the value of the task/process, or if it set after comparing the requested value with the max_memory threshold?pontus
02/28/2024, 10:36 AMmax_memory should be 240, otherwise nextflow (the nf-core configurations) would limit all tasks to 112.
(Actually, I'd assume task.mem is not available when max_memory is calculated as well, but still.)Pieter Moris
02/28/2024, 11:36 AMPhil Ewels
02/29/2024, 7:24 AMclusterOptions = { check_max(task.mem) >= 112.GB ? "--constraint=mem256" : "--constraint=mem128"}Phil Ewels
02/29/2024, 7:25 AMcheck_max is this function (depending on where you're setting this config, you may find that you need to copy + paste this function into that config file, if it doesn't work)Phil Ewels
02/29/2024, 7:25 AMparams.max_memoryPhil Ewels
02/29/2024, 7:26 AMPhil Ewels
02/29/2024, 7:26 AMcheck_max the max_memory params doesn't do anything anyway 😅pontus
02/29/2024, 7:27 AMtask.mem will already be limited because process.memory uses check_max in nf-core.Phil Ewels
02/29/2024, 7:28 AMprocess.clusterOptions and not process.memoryPhil Ewels
02/29/2024, 7:28 AMPhil Ewels
02/29/2024, 7:28 AMPhil Ewels
02/29/2024, 7:28 AMPhil Ewels
02/29/2024, 7:29 AMPieter Moris
03/01/2024, 10:47 AMconstraint option does not seem to work on our HPC 😞 Will reach out to the support team to let them know. Apparently I can access the high memory nodes via a different partition name though, so I ended up just adding additional profiles for high mem nodes.
Thanks for the info on the check_max function though!pontus
03/01/2024, 11:30 AM