A community effort to collect a curated set of analysis pipelines built using Nextflow.

nf-core

I have been running my jobs on an HPC system with max job runtime = 3 days. currently I've got a problem where the parent job has been running for 48hrs but the child jobs (GTDB-TK) are waiting in the queue.
Last time this happened the parent job failed after hitting the runtime wall, but the child jobs kept running (GTDB-TK),  i let them run but eventually, when i resumed the mag pipeline after the child jobs had finished, the jobs that had finished slightly before (PROKKA) and anything after (GTDBK) the parent job had died, were not cached. Is there a way around this?
<#CE9MS66BS>

Hi! I don't think so. If Nextflow is killed, it will not track the completion of tasks while it's not running.

For HPC I guess the common way to do it would be to run Nextflow in the head node using screen/tmux so the process keep running and you don't lose access to the outupt.

If you can't do that in your HPC, then I would run it first with GTDB-Tk disabled, because that requires a lot of time, and then run again with GTDB-Tk.

Hi Diego thank you very much for your help!