@Maxime Garcia this conditional linking is still giving me some errors when using an S3 workDir sad cowboy https://github.com/nf-core/methylseq/blob/aac172df2c1c6558683885d3d52157bed3b6bbab/modules/nf-core/bwameth/align/main.nf#L32-L34

touch -c BwamethIndex/*
  
  if [ ! -f BwamethIndex/genome.fa ]; then
      ln -s $(readlink genome.fa) BwamethIndex/genome.fa;
  fi
  
  bwameth.py \
       \
       \
      -t 4 \
      --reference BwamethIndex/genome.fa \
      SRR389222_sub2_trimmed.fq.gz \
      | samtools view  -@ 4 -bhS -o SRR389222_sub2.bam -

error:

Command error:
  ln: genome.fa: File exists

Where is this failing?

chatgpt's response The issue here lies in the ln -s $(readlink genome.fa) BwamethIndex/genome.fa; line. Even though you check for the existence of the genome.fa file within the BwamethIndex directory, if genome.fa is already a link or exists as a file within the same directory, this command will fail since ln -s does not overwrite existing files or links by default. To fix this, you can modify the logic to either delete any existing link/file before creating the symlink or adjust the condition. Here’s an improved approach that checks if BwamethIndex/genome.fa does not already exist before creating the symlink:

if [ ! -e BwamethIndex/genome.fa ]; then
  ln -s $(readlink genome.fa) BwamethIndex/genome.fa
fi

The -e flag checks for both files and symbolic links, so this will prevent the ln command from being executed if BwamethIndex/genome.fa already exists in any form. This should resolve the File exists error.

interesting solution by chatGPT

ya im trying now

Nothing I can think off at the moment

ok updating modules and patch release

🙏

Hi! I am getting the same error but I am not using an

S3 workDir

(I am running the pipeline using CPU (and not GPU) on the Finnish IT Center for Science servers which, as far as I can tell, do not use Amazon S3 storage service). Interestingly, I started a batch of 10 samples a couple days ago and the pipeline completed successfully without this error. But today, I started a new batch of ~50 samples, and I got this error. It seems that this error happened only for [at least] one sample and not all (but I am not 100% sure, this is because, if I check a the

work

subdirectory for a running

BWAMETH_ALIGN

process (despite pipeline failure), I can see that the BAM file is increasing in size). Any ideas how to address this issue?

Interesting

I am using

<https://github.com/nf-core/methylseq> [confident_legentil] DSL2 - revision: 4415e90a60 [3.0.0]

with

N E X T F L O W   ~  version 24.10.3

. • 7 out of 45 samples have this issue. The rest went just fine. • I think we can rule out any specific node as a cause for this. Because, I can see that these seven jobs had failed on different nodes. And at least there is one

BWAMETH_ALIGN

job running for another sample on the same node, that the other job failed.