Hi I am facing the following error when running the scnanoseq pipeline on my computing cluster.

Caused by:
  Process `NFCORE_SCNANOSEQ:SCNANOSEQ:BLAZE (t4_shCTRL)` terminated with an error exit status (1)

Command executed:

  main.py \
      --expect-cells 10000 \
      --full-bc-whitelist /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/3M-february-2018.zip \
      --output-prefix t4_shCTRL. \
      --threads 6 \
      --kit-version 3v3 \
      $(pwd)
  
  
  tail -n +2 t4_shCTRL.putative_bc.csv > t4_shCTRL.putative_bc.no_header.csv
  
  cat t4_shCTRL.putative_bc.no_header.csv | \
      cut -f2 -d',' | \
      sort -T $(pwd) | \
      uniq -c | \
      awk '{print $2","$1}'> t4_shCTRL.bc_count.txt
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_SCNANOSEQ:SCNANOSEQ:BLAZE":
      blaze: 2.2.0
  END_VERSIONS

Command exit status:
  1

Command output:
  [93mYou are using 3M-february-2018.zip as the full barcode whitelist. Note that the barcodes not listed in the file will never be found.[0m
  [91mError: can not find /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/3M-february-2018.zip[0m

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Matplotlib created a temporary cache directory at /tmp/matplotlib-0bvxxktx because the default path (/home/tmhaxs421/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  [93mYou are using 3M-february-2018.zip as the full barcode whitelist. Note that the barcodes not listed in the file will never be found.[0m
  [91mError: can not find /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/3M-february-2018.zip[0m
  tail: cannot open 't4_shCTRL.putative_bc.csv' for reading: No such file or directory

Work dir:
  /condo/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/scripts/work/b9/bfbfff130bd36eac64bfcf82178f88

My file for the --full-bc-whitelist exists but the error mentions that it is not found. Can someone help me with troubleshooting this issue please. I will really appreciate it. Thanks in advance

Hi Aravind, would you mind also sending the code you are using to run the pipeline? (Additionally any parameter that are being used)

Yes, of course. This is the exact code along with the flags, that I am using to run the nextflow pipeline.

module load nextflow/23.09.1   
module load java/corretto/17.0.8.8.1


nextflow run nf-core/scnanoseq \
  --input /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/scripts/samplesheet.csv \
  --outdir /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/results \
  --fasta /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/genome.fa \
  --gtf /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/genes.gtf \
  --barcode_format 10X_3v3 \
  --whitelist /home/tmhaxs421/brannanlab/tmhaxs421/STAMP/LongRead_Nextflow_scnanoseq/reference/3M-february-2018.zip \
  -profile singularity \
  -r dev

Oh actually, just because I notice the filename. Are you trying to use the 10X 3' barcode whitelist? If so, the pipeline should use that whitelist when the

10X_3V3

barcode format is set, which looks like you do have in your script, so you don’t even need to set the whitelist parameter

Yes I had tried that method where I did not specify the barcode whitelist from my local drive. I faced the same error, where the nextflow pipeline was not able to find that white list as it was searching for it in the wrong location. I have deleted that log file so I am not able to provide it here. Should I try removing the --barcode-format flag and just specify the --whitelist flag? (edit) --barcode-format is a required input, so the above does not work

Oh interesting, I am also able to reproduce it. I’ll see if I can track down the issue and push a fix for it

@Aravind Sundaravadivelu: thanks for reporting this. We believe we found the source of the issue and Austyn will be patching a fix soon. Thanks again!

Hi Aravind, we’ve pushed a potential fix for the issue. If you wouldn’t mind trying again with the latest dev, that should hopefully resolve the whitelist issue. I’ve been running the test profile as well and hadn’t seen any other issues, but please let us know if you encounter anything else

@Aravind Sundaravadivelu: I would also advise (especially since it is still in dev although close to release), to first run the

test

profile if you can. This would ensure that hopefully the pipeline is indeed behaving as expected before you re-start your new run. The

test

profile should not take long to run. Let us know how it goes. Thanks!

Thank you, @Austyn Trull and @Lara Ianov, for the quick fix. I’ll run the test profile and let you know if it works on my end. This pipeline is one of the closest I've found to what I need, and I’m excited to see it release soon.

@Aravind Sundaravadivelu: Just wanted to give you an update on our end regarding the pipeline at this stage. We just made some changes in the latest PR following formal review from nf-core along with a few other improvements from our end. One important change here is that we have upgraded

IsoQuant

to the latest version (3.5.0). We were in communication with the author regarding a few memory issues which are now resolved but on the plus side this version brings significant improvements to transcript-level quantification. An FYI since we believe this change is quite important. Also on our end, we hope we are ready for release pending any other needs from nf-core. So what you see there is either the "final" pipeline (as in first release) or very close to final pending any other needs from nf-core or if a user reports an issue.