Can I increase the running time. i have 4 samples in my sample sheet. each file size is ~ 50GB including R1 and R2.

ERROR ~ Error executing process > 'NFCORE_HIC:HIC:HICPRO:HICPRO_MAPPING:BOWTIE2_ALIGN (Patz1)'

Caused by:
  Process exceeded running time limit (16h)


Command executed:

  INDEX=`find -L ./ -name "*.rev.1.bt2" | sed "s/\.rev.1.bt2$//"`
  [ -z "$INDEX" ] && INDEX=`find -L ./ -name "*.rev.1.bt2l" | sed "s/\.rev.1.bt2l$//"`
  [ -z "$INDEX" ] && echo "Bowtie2 index files not found" 1>&2 && exit 1
  
  bowtie2 \
      -x $INDEX \
      -U HiC_Patz1_7dpi_1.fastq.gz \
      --threads 12 \
      --un-gz Patz1_0_R1.unmapped.fastq.gz \
      --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder \
      2> Patz1_0_R1.bowtie2.log \
      | samtools view -F 4 --threads 12 -o Patz1_0_R1.bam -
  
  if [ -f Patz1_0_R1.unmapped.fastq.1.gz ]; then
      mv Patz1_0_R1.unmapped.fastq.1.gz Patz1_0_R1.unmapped_1.fastq.gz
  fi
  
  if [ -f Patz1_0_R1.unmapped.fastq.2.gz ]; then
      mv Patz1_0_R1.unmapped.fastq.2.gz Patz1_0_R1.unmapped_2.fastq.gz
  fi
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_HIC:HIC:HICPRO:HICPRO_MAPPING:BOWTIE2_ALIGN":
      bowtie2: $(echo $(bowtie2 --version 2>&1) | sed 's/^.*bowtie2-align-s version //; s/ .*$//')
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
      pigz: $( pigz --version 2>&1 | sed 's/pigz //g' )
  END_VERSIONS

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /media/manoj/Data/Manoj/Hi-C/nf-core-hic_2.1.0/2_1_0/work/cb/4acb2e3826060caafcb160ac0a2d00

Container:
  quay.io/biocontainers/mulled-v2-ac74a7f02cebcfcc07d8e8d1d750af9c83b4d45a:a0ffedb52808e102887f6ce600d092675bf3528a-0

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

Yup, docs here: https://nf-co.re/docs/usage/getting_started/configuration#tuning-workflow-resources

Thanks, @Phil Ewels. I’ve updated my config file. I’ll check and get back to you if I encounter any issues.

Great! You only need to specify

time

if that's all you want to overwrite

Four of my samples have been running for more than 6 days. Should it take this much time, or is there an issue with my data? When I used

htop

, the Hi-C pipeline was utilizing 100% of my 32 cores and 260 GB of RAM. However, in 6 days, it managed to complete the alignment for only one sample.

That's at the long end of what's reasonable, but it's not totally unbelievable, especially if these are deeply sequenced samples. HiC data can be huge.

@Phil Ewels tried searching for *.command.log file. I can't find anything.

The

.command.log

files are in each work directory. See for details. Here your pipeline hasn't failed, but the nextflow log output shows the task hash. So you can

cd work/8d/c59423...

to get into the work directory for the running alignment task. That's where that job is running right now, and where you'll be able to tail the

.command.log

file.

Thanks found it the log file is at 0 bytes tail is still loading. I'm also in a confused state. because

htop

says bowtie is running.

ls -la

to show hidden files too

Yup, the .common.log file is also empty.

Ok then something smells fishy

shall I kill the process or wait for it to get completed.

Ah really? Ok well that's a good sign!

It has managed 16gb of output file in 2 days?

I’m using an ssd only which has a write speed on 800mb. I ran bowtie aligner separately for the same file which took around 4 hours to complete @Simon Pearce. I have no clue why this is taking this much time.