Hi team - I am running into an issue with time-out...
# help
u
Hi team - I am running into an issue with time-out for large samples with the RNA-seq pipeline. The pipeline is failing, but I can't seem to figure out why from the report or timeline. Would anyone be able to provide advice? The error output is "SIGHUP". I have attached log, execution timeline and report. I modified the config to include the following as I was getting time out with some of the larger samples:
Copy code
process {
  withName: '.*' {
    time = null
  }
}
Some of the samples have undergone deep sequencing - 50gb for each forward and reverse. Let me know if you need more information. Thanks so much.
#️⃣ 1
c
Hi, @User! Slack uses channels to organise discussions. Press the
...
more
option on the left sidebar. Then hover over Your organisation, and finally select Channels to browse. Each nf-core pipeline has it’s own channel (e.g. #CE8SSJV3N). Please post any questions about specific pipelines in the relevant channel. If you’re not sure where to post your message, please ask @Igor Trujnara for help.
p
It's not clear for me from your description what happens (e.g. do your jobs get
SIGHUP
or is that the main nextflow process - I didn't spend much time but it kind of looks like the latter in the log provided). Are you running on a cluster?
SIGHUP
is typically used to provide graceful termination before cancelling a job that times out. If so, increasing the time nextflow will let process run doesn't matter if the job is torn down by the scheduler, rather you need to increase the time requested (or run repeatedly with
-resume
).
u
Hi @pontus - thanks so much for your response. I am running it on a server that I have spun up, so there is no scheduler. The SIGHUP is from the main nextflow process. I am going to try running the pipeline with a subset of samples to see if it is just the large samples causing issues!
p
"spun up" sounds like it's not your desktop. Do you run your stuff under `screen`/`tmux`? Otherwise you are likely to see this if there's a network hiccup.
➕ 1
u
Hi - sorry for not being clear. Yes running on a server through the Nectar Research Cloud. I was running under tmux. It was a time-out error based on one sample, then a out of memory error for another sample (still working on that sample at the moment). I changed the nextflow.config file to include the following: process { withName: '.*' { time = null } } process { withName: 'NF_CORE_RNASEQRNASEQALIGN_STAR:STAR_ALIGN' { time = null } } process { withName: 'NFCORE_RNASEQRNASEQBAM_DEDUP_UMI_STARBAM DEDUP STATS SAMTOOLS UMITOOLS GENOMEUMITOOLS_DEDUP.*' { memory = 160.GB cpus = 50 time = '96.h' } } I was running tmux and with docker. I think the umitools dedup still isn't quite right, because it isn't using more than 6gb memory to process the sample, but will update again if I get any further solutions.