Luuk Harbers
12/19/2024, 3:00 PM--account that is set through an env variable or use a different one.
A small snippet of this is:
// Define the queue closure
queue = {
task.memory >= 239.GB ?
(task.time >= 72.h ? 'dedicated_big_bigmem' : 'bigmem,hugemem') :
(task.time >= 72.h ? 'batch_long,batch_icelake_long,batch_sapphirerapids_long' : 'batch,batch_sapphirerapids,batch_icelake')
}
// Set clusterOptions, changing account based on queue
clusterOptions = queue =~ /dedicated/ ? "--clusters=wice --account=lp_big_wice_cpu" : "--clusters=wice --account=$tier1_project"
So basically, if 'dedicated' is in the queue name, it should take lp_big_wice_cpu else it should take the $tier1_project variable. However when I try this, even if my queue contains 'dedicated' it always takes the latter option. Can someone spot a clear mistake in this or am I misunderstanding something perhaps?
For example in a .command.run script I can see it takes (lp_lvibccbeu is the env variable account):
#SBATCH -p dedicated_big_bigmem
#SBATCH --clusters=wice --account=lp_lvibccbeupontus
12/19/2024, 3:20 PMqueue is uncontroversial.
As for clusterOptions. If you don't make it a closure somehow, it will only be called once (whenever the file is parsed, I assume).
I don't know (and can't be bothered to check myself right now) if accessing other directives (queue). work like this. I have a fairly strong preference for putting the decision tree in a function and having the closure being only a call with the desired inputs.
And if the access works as one could possibly expect, I believe you'd see a closure rather than the result of the evaluation, you likely need to call the closure to get the result.Luuk Harbers
12/19/2024, 4:38 PMclusterOptions as a closure would make it so that it is dynamically evaluated.
So this 'should' work:
clusterOptions = {
queue =~ /dedicated/ ? "--clusters=wice --account=lp_big_wice_cpu" : "--clusters=wice --account=$tier1_project"
}
However, it ends up giving me the same it gives me the option for the False evaluation even if the queue contains 'dedicated'..pontus
12/19/2024, 5:20 PMqueue at all, I would expect it to yield the closure. To get the value you'd need to evaluate it if that's what's happening.Luuk Harbers
12/19/2024, 5:24 PMThomas Danhorn
01/03/2025, 7:15 PMqueue (I think I had to do this for other things inside closures)? Try ... *process*.queue =~ /dedicated/ ? ....Luuk Harbers
01/06/2025, 8:56 AMprocess.queue does not seem to change anything. It just seems that my expression
process.queue =~ /dedicated/ ? "--clusters=wice --account=lp_big_wice_cpu" : "--clusters=wice --account=$tier1_project"
Always returns False. Even though I can see in the .command.run file the final partition contains dedicated.
#SBATCH -p dedicated_big_bigmem
I'm not familiar with groovy at all so maybe I am simply doing something wrong with the expression?pontus
01/06/2025, 9:35 AMqueue never appears.Thomas Danhorn
01/06/2025, 6:56 PMqueue is set (e.g. statically, rather than dynamically)? You could see if you can debug this with print statements, or somethings, or try to find some work-around like Pontus suggests.
If you can nail this down to some regex-evaluation issue that is reproducible (maybe even with things other than queue), it moves from being a #CGRTMASKY issues to something more general, and you might get more attention to that over in the Nextflow Slack, e.g. in #nextflow-help. 🤞Luuk Harbers
01/07/2025, 8:52 AMqueue value within the clusterOptions{} enclosure and then it works! Thanks for all the suggestions everyone!