Third, you (<@UGJEZTL4D>) comment that "Due to <ho...
# configs
d
Third, you (@pontus) comment that "Due to how partitions are set up on dardel, in particular the lack of long-runtime nodes with more memory. Some runs may be difficult to get through." (https://nf-co.re/configs/pdc_kth/) but is this true? If one asks for up to 2 TiB RAM one should be able to get seven days, right? (Indeed shorter than Uppmax's 10 days, but still quite long.)
p
I think this was mostly that getting your jobs through the queue could be an ordeal, but yes, looking at
memory
now it seems you can run large jobs quite long.
Copy code
pontusf2@login1:~$ scontrol  show part memory
PartitionName=memory
   AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL
   AllocNodes=ALL Default=NO QoS=N/A
   DefaultTime=01:00:00 DisableRootJobs=YES ExclusiveUser=NO GraceTime=0 Hidden=NO
   MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED MaxCPUsPerSocket=UNLIMITED
   Nodes=nid[000101-000118,001772-001779,002552-002567]
   PriorityJobFactor=10 PriorityTier=10 RootOnly=NO ReqResv=NO OverSubscribe=EXCLUSIVE
   OverTimeLimit=NONE PreemptMode=OFF
   State=UP TotalCPUs=10752 TotalNodes=42 SelectTypeParameters=NONE
   JobDefaults=(null)
   DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED
   TRES=cpu=10752,mem=48612416M,node=42,billing=10752

pontusf2@login1:~$ scontrol  show node nid[000101-000118,001772-001779,002552-002567] | grep RealMem | sed -e 's/Alloc.*//' | sort -u
   RealMemory=1008696 
   RealMemory=1812960 
   RealMemory=494348 
pontusf2@login1:~$
d
Thanks! What I can't see in the config is that you set
--mem
except if the user asks for <=6.GB or none at all. One has to, to get a correctly sized
memory
node, right (e.g.
-p memory --mem=1200G
)? Or is this set somewhere else? (I don't see that the number of cores is set either, so I'm assuming I'm missing something.)
p
The way this profile is done is it only supplies extra arguments to slurm to fix things if needed. So
cpus
and
memory
are supplied by nextflow (and limited by
process.resourceLimits
), and then the profile relies on slurm handling to override if there's a bad memory setup.
d
I see, thanks for the explanation. Then I'll make a PR right away.