Zachary Henning
07/02/2024, 1:02 AMprocess {
executor = 'slurm'
queue = { task.time <= 2.d ? 'epyc-64' : 'largemem'}
clusterOptions = '--account=thrash_89 --export=NONE --mail-type=END,FAIL --mail-user=zjhennin@usc.edu'
maxRetries = 3
errorStrategy = { task.attempt <=3 ? 'retry' : 'finish' }
cache = 'lenient'
}
singularity {
enabled = true
autoMounts = true
}
executor {
queueSize = 5000
submitRateLimit = '10 sec'
}
profiles {
largemem {
params {
config_profile_description = 'University of Southern California HPC profile provided by nf-core/configs.'
config_profile_contact = 'carc-support@usc.edu'
config_profile_url = 'https://www.carc.usc.edu/'
max_memory = 1000.GB
max_cpus = 24
max_time = 7.d
}
}
epyc-64 {
params {
config_profile_description = 'University of Southern California HPC profile provided by nf-core/configs.'
config_profile_contact = 'carc-support@usc.edu'
config_profile_url = 'https://www.carc.usc.edu/'
max_memory = 248.GB
max_cpus = 64
max_time = 2.d
}
}
}
Is this appropriate to use profiles in order to define each partitions max memory, cpus, and time? Also, I see the tutorial allows the choosing of partitions based on CPUs (like he time argument I have in the config file), can it also do so with memory? i.e. something like
queue = { task.memory <= 200.GB ? 'epyc-64' : 'largemem'}pontus
07/02/2024, 5:43 AMZachary Henning
07/02/2024, 5:49 AMZachary Henning
07/02/2024, 5:53 AMJames Fellows Yates
07/02/2024, 7:26 AMJames Fellows Yates
07/02/2024, 7:28 AMqueue = { time < 7.d && memory >= 512.GB ? 'huge' : time <7.d && memory < 512.GB ? 'big' : time <= 24.h ? 'short' : 'default' }James Fellows Yates
07/02/2024, 7:28 AMJames Fellows Yates
07/02/2024, 7:28 AMpontus
07/02/2024, 8:35 AMprocess.memory in clusterOptions (e.g. if it's too small or too big).
One approach would be to set the values to what works on either partition, if that works then good.
Another is to announce the maximum and do something sensible once the requests are known.
Or something in between.
If it's for running a specific pipeline, you can probably get by using one partition and associated values as default and tuning the processes as needed.Zachary Henning
07/02/2024, 6:25 PMprocess {
executor = 'slurm'
queue = { time, memory ->
if (memory > 240.GB && time < 7.d) {
'largemem'
} else if (time < 2.d || memory <= 240.GB) {
'epyc-64'
} else {
'largemem'
}
}
clusterOptions = '--account=thrash_89 --export=NONE --mail-type=END,FAIL --mail-user=zjhennin@usc.edu'
maxRetries = 3
errorStrategy = { task.attempt <= 3 ? 'retry' : 'finish' }
cache = 'lenient'
}
So something like this should be sufficient instead of using the profilesZachary Henning
07/02/2024, 6:37 PMpontus
07/02/2024, 7:20 PMtask.James Fellows Yates
07/02/2024, 7:22 PMtask.memoryZachary Henning
07/02/2024, 7:27 PMprocess {
executor = 'slurm'
queue = { task.time, task.memory ->
if (memory > 240.GB && time < 7.d) {
'largemem'
} else if (time < 2.d || memory <= 240.GB) {
'epyc-64'
} else {
'largemem'
}
}
clusterOptions = '--account=thrash_89 --export=NONE --mail-type=END,FAIL --mail-user=zjhennin@usc.edu'
maxRetries = 3
errorStrategy = { task.attempt <= 3 ? 'retry' : 'finish' }
cache = 'lenient'
}
with this adjustment I now get the error that it can't parse the config fileZachary Henning
07/02/2024, 7:31 PMqueue = { task.time < 7.d && task.memory >= 512.GB ? 'largemem' : task.time <7.d && task.memory < 512.GB ? 'epyc-64' : task.time <= 24.h ? 'epyc-64' : 'largemem' }
Thank you so much @pontus and @James Fellows Yates for helping a beginner like myself!!