Francisca Castillo
03/30/2024, 3:22 PMThomas Danhorn
03/31/2024, 6:29 AM-profile docker might be sufficient for most of the configuration settings, but you will still have to tell the pipeline the limits of your machine by specifying the parameters --max_memory (probably 62.GB in your case, --max_time (this is mostly relevant for systems with schedulers, which kill processes that go over time), and --max_cpus (number of CPU cores in your system; you could subtract one or two for the Nextflow process itself, especially if have a lot). If you don't know the number of CPU cores, run this:
cpus=$(grep '^physical id' /proc/cpuinfo | sort -u | wc -l)
corespercpu=$(grep -m 1 '^cpu cores' /proc/cpuinfo | grep -oP '\d+$')
echo $((cpus * corespercpu))
If you have a system with multiple nodes and a scheduler, you may want to add additional configuration to take advantage of it (the default local executor only runs on the machine where you start the pipeline).
If a process needs more memory than your machine can provide, you will have to either find a way through the pipeline that does not use that process (depending on what it is, there may not be one) or find a different system to run to pipeline. ¡Buena suerte!