Hi, I am currently running sarek pipeline for WGS data on Dardel using the default KTH profile. While the main job runs within a reasonable time, it submits multiple sub-jobs that remain in the queue indefinitely until the main job's wall time expires. I have tried to decrease the resourceLimits parameters in the config file, but I got the same issue. The NODELIST(REASON) in the squeue --me show "Priority" status for all jobs. I would appreciate any guidance on resolving this. Thanks

I usually run the

nextflow run ... -profile pdc_kth ...

command in a screen session. This way the

nextflow

process doesn't run out of time

so but why the sub-jobs are not starting? usually I set the wall-time around 72 hours which is enough for the samples depth I have

Are you over time quota? check

projinfo

yes, I am over the quota, is there recommended resources limit that I can set to let my jobs run on the less priority queues?

You can look at the reports in the

pipeline_info

folder and see which processes are not using all the cpus or memory and then scale them down, but other than that there's not much to do but wait

I see, thanks for the tips

There's not really any specific queues for lesser priority, you're just competing when there's room against others. But they will start eventually. I typically recommend the Mahesh approach of running in a screen/tmux instead of in a job for the monitoring process (it's very lightweight).