https://nf-co.re logo
Join Slack
Powered by
Dear all, the pipeline results as complete but I c...
# rnavar
l
Dear all, the pipeline results as complete but I cannot find the snpeff output, is this problem common ? Basically the last output produced is the variant_calling original and filtered. I post the last 7 lines of the log
Copy code
-[nf-core/rnavar] Pipeline completed successfully-
Completed at: 29-Nov-2025 05:47:16
Duration    : 10h 38m 48s
CPU hours   : 73.3 (0.4% failed)
Succeeded   : 77
Failed      : 1
r
PLease post moreinfo : nextflow.log & terminal output if you have it
s
To help we'd love a few more details please: 1. The Nextflow command you used to launch the pipeline 2. Any custom Nextflow config files 3. The full error message 4. The corresponding 
.nextflow.log
file, where you are running the analysis. Remember to use triple backslashes on their own lines for code blocks. Also take a look at https://nf-co.re/usage/troubleshooting for more information on debugging failed pipelines. Thanks!
l
Copy code
cat run_nfcore_rnavar2_q.sh 
#!/bin/bash

LOGFILE="/data/myname/projects/rnavar_test/logs/rnavar_run_$(date +%Y%m%d_%H%M%S).log"

nextflow run nf-core/rnavar \
  -profile singularity \
  -c /data/myname/bins/custom.config2 \
  --input /data/myname/projects/rnavar_test/samplesheet_test.csv \
  --outdir /data/myname/projects/rnavar_test/results_test \
  --genome Olea_europaea \
  --skip_baserecalibration \
  &> "$LOGFILE"
Copy code
cat custom.config2 
process {

  // Fix newline warning in process.shell
  shell = 'bash -euo pipefail'

  // Default resources
  cpus = 8
  memory = '28 GB'
  beforeScript = 'chmod +x .command.run || true'

  // Labels override
  withLabel: process_low {
    cpus = 4
    memory = '8 GB'
  }

  withLabel: process_medium {
    cpus = 6
    memory = '16 GB'
  }

  withLabel: process_high {
    cpus = 8
    memory = '28 GB'
  }

  withLabel: process_long {
    cpus = 8
    memory = '28 GB'
  }

  // STAR genomeGenerate needs more memory
  withName: 'NFCORE_RNASEQ:PREPARE_GENOME:STAR_GENOMEGENERATE' {
    cpus = 8
    memory = '28 GB'
  }

  // fastp combined with UMI-tools & QC inference can spike RAM usage
  withName: 'NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FASTQ_FASTQC_UMITOOLS_FASTP:FASTP' {
    cpus = 6
    memory = '16 GB'
  }
}

executor {
  queueSize = 2
  maxForks = 2
}

params {
  genome = 'Olea_europaea'

  genomes {
    'Olea_europaea' {
      fasta      = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.dna.toplevel.fa'
      gtf        = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.62.gtf'
      star_index = '/data/myname/refs/STAR/Olea_europaea'
      snpeff {
        cache = '/data/myname/bins/snpEff/data'
      }
    }
  }
}
I post the script and the config file i've used
r
you need to set it with 
--tools
: https://nf-co.re/rnavar/1.2.1/parameters/#tools cc @Maxime Garcia tools parameter for some reason is hidden, maybe we should unhide it to make it more prominent
💯 1
l
sorry I dont get it
r
you need to add 
--tools snpeff
to enable the snpeff run
l
Thanks, I launched it:
Copy code
Nextflow 25.10.2 is available - Please consider updating your version to it

 N E X T F L O W   ~  version 25.04.0

Launching `<https://github.com/nf-core/rnavar>` [scruffy_fourier] DSL2 - revision: 2cbf046936 [master]


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rnavar 1.2.1
------------------------------------------------------
Input/output options
  input                 : /data/myname/projects/rnavar_test/samplesheet_test.csv
  outdir                : /data/myname/projects/rnavar_test/results_test
  tools                 : snpeff

Variant filtering
  gatk_vf_qd_filter     : 2.0
  gatk_vf_fs_filter     : 30.0

Pipeline stage options
  skip_baserecalibration: true

Reference genome options
  genome                : Olea_europaea
  fasta                 : /data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.dna.toplevel.fa
  gtf                   : /data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.62.gtf

Generic options
  trace_report_suffix   : 2025-12-02_10-46-00

Core Nextflow options
  revision              : master
  runName               : scruffy_fourier
  containerEngine       : singularity
  launchDir             : /data/myname/bins
  workDir               : /data/myname/bins/work
  projectDir            : /home/zephyr/.nextflow/assets/nf-core/rnavar
  userName              : zephyr
  profile               : singularity
  configFiles           : /home/zephyr/.nextflow/assets/nf-core/rnavar/nextflow.config, /data/myname/bins/custom.config2

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
* The pipeline
    <https://doi.org/10.5281/zenodo.6669636>

* The nf-core framework
    <https://doi.org/10.1038/s41587-020-0439-x>

* Software dependencies
    <https://github.com/nf-core/rnavar/blob/master/CITATIONS.md>

[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_FASTA                   -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GATK4_CREATESEQUENCEDICTIONARY -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_GTF                     -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GFFREAD                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GTF2BED                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:REMOVE_UNKNOWN_REGIONS         -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:SAMTOOLS_FAIDX                 -

[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_FASTA                   -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GATK4_CREATESEQUENCEDICTIONARY -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_GTF                     -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GFFREAD                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GTF2BED                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:REMOVE_UNKNOWN_REGIONS         -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:SAMTOOLS_FAIDX                 -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:UNTAR                          -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:STAR_INDEXVERSION              -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:STAR_GENOMEGENERATE            -
This path is not available within annotation-cache.
Please check <https://annotation-cache.github.io/> to create a request for it.


[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_FASTA                   -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GATK4_CREATESEQUENCEDICTIONARY -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GUNZIP_GTF                     -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GFFREAD                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:GTF2BED                        -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:REMOVE_UNKNOWN_REGIONS         -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:SAMTOOLS_FAIDX                 -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:UNTAR                          -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:STAR_INDEXVERSION              -
[-        ] process > NFCORE_RNAVAR:PREPARE_GENOME:STAR_GENOMEGENERATE            -
I dont understand what's wrong
r
this configuration looks off:
Copy code
params {
  genome = 'Olea_europaea'

  genomes {
    'Olea_europaea' {
      fasta      = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.dna.toplevel.fa'
      gtf        = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.62.gtf'
      star_index = '/data/myname/refs/STAR/Olea_europaea'
      snpeff {
        cache = '/data/myname/bins/snpEff/data'
      }
    }
  }
}
@Maxime Garcia do you know if you can configure genome blocks as above. i didn't think you could, but happy to be corrected In general it's not a good idea to provide params in config fi;es to avoid any precedence issues.
l
On top of it, as Olea_europaea is present in the index of snpeff, I'm not sure how to behave. But let's say It's not present. I've actually run the database build like this:
Copy code
cd ~/snpEff
java -jar snpEff.jar build -gtf22 -v Olea_europaea
and now im giving that path to the data folder.
m
Oh I see, so it's a custom annotation not one in snpeff
I would try something like:
Copy code
params.snpeff_db         = 'Olea_europaea'
params.snpeff_cache      = '/data/myname/bins/snpEff/data'
l
Thanks Ill update you asap
Untitled
Untitled
m
ok, so from what I can see, it was made with 5.3, but not compatible with the rnavar shipped 5,1
l
I posted the log after running the updated custom.config
Copy code
process {

  // Fix newline warning in process.shell
  shell = 'bash -euo pipefail'

  // Default resources
  cpus = 8
  memory = '28 GB'
  beforeScript = 'chmod +x .command.run || true'

  // Labels override
  withLabel: process_low {
    cpus = 4
    memory = '8 GB'
  }

  withLabel: process_medium {
    cpus = 6
    memory = '16 GB'
  }

  withLabel: process_high {
    cpus = 8
    memory = '28 GB'
  }

  withLabel: process_long {
    cpus = 8
    memory = '28 GB'
  }

  // STAR genomeGenerate needs more memory
  withName: 'NFCORE_RNASEQ:PREPARE_GENOME:STAR_GENOMEGENERATE' {
    cpus = 8
    memory = '28 GB'
  }

  // fastp combined with UMI-tools & QC inference can spike RAM usage
  withName: 'NFCORE_RNASEQ:RNASEQ:FASTQ_QC_TRIM_FILTER_SETSTRANDEDNESS:FASTQ_FASTQC_UMITOOLS_FASTP:FASTP' {
    cpus = 6
    memory = '16 GB'
  }
}

executor {
  queueSize = 2
  maxForks = 2
}

params {
  genome = 'Olea_europaea'
  params.snpeff_db         = 'Olea_europaea'
  params.snpeff_cache      = '/data/myname/bins/snpEff/data'
  genomes {
    'Olea_europaea' {
      fasta      = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.dna.toplevel.fa'
      gtf        = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.62.gtf'
      star_index = '/data/myname/refs/STAR/Olea_europaea'
    }
  }
}
m
Could you try the 
dev
branch, I am about to release, and I think it should work
l
ok sure, please when youre ready, tell me how
m
Actually we have 2 simple solutions: 1. run the dev branch by replacing 
-r  1.2.1
with 
-r dev -latest
on your nextflow command. you might need to update nextflow, but this should do the trick: 
export NXF_VER=25.10.0 ; nextflow self-update
2. Keep the same command but add this to your custom config file to overwrite the container and use the latest snpeff instead.
Copy code
process {
    withName:SNPEFF_SNPEFF {
        container = '<https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/37/37a5be8afdc2e112fd1bb4aa226e009a437e6537f50a51ed909fe2018fd62e99/data>'
    }
}
Maybe you'd rather do the second option, as I think it changes less the pipeline itself
l
should i have docker installed to rerun the pipeline like this ? So far im using -profile singularity because I have it
r
singularity will convert the docker image on the fly so no need. Or you can go and find the singularity image here
(in general containers are build here: and you can get the docker or singularity container directly: https://seqera.io/containers/?packages=bioconda::snpeff=5.3.0a
m
yes, that was the singularity image
I noticed Luca was running on singularity, so I gave the singularity image 😎
l
Ook great thanks
ill repost the config in a moment
Copy code
process {
  shell = 'bash -euo pipefail'
  cpus = 8
  memory = '28 GB'
  beforeScript = 'chmod +x .command.run || true'

  withLabel: process_low {
    cpus = 4
    memory = '8 GB'
  }

  withLabel: process_medium {
    cpus = 6
    memory = '16 GB'
  }

  withLabel: process_high {
    cpus = 8
    memory = '28 GB'
  }

  withLabel: process_long {
    cpus = 8
    memory = '28 GB'
  }

  // PATCH: Use latest SnpEff container
  withName: SNPEFF_SNPEFF {
    container = '<https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/37/37a5be8afdc2e112fd1bb4aa226e009a437e6537f50a51ed909fe2018fd62e99/data>'
  }
}

executor {
  queueSize = 2
  maxForks = 2
}

params {
  genome = 'Olea_europaea'
  params.snpeff_db         = 'Olea_europaea'
  params.snpeff_cache      = '/data/myname/bins/snpEff/data'

  genomes {
    'Olea_europaea' {
      fasta      = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.dna.toplevel.fa'
      gtf        = '/data/myname/refs/genomes/Olea_europaea/Olea_europaea.OLEA9.62.gtf'
      star_index = '/data/myname/refs/STAR/Olea_europaea'
    }
  }
}
✅ 1
m
looks ok to me
l
another naive question: why if i rerun the pipeline it doesnt continue from the last step ? it always restarts from scratch
r
In your original command, I don't see 
-resume
Did you set that on reruns?
party this 1
l
ok cool so like that:
Copy code
#!/bin/bash

LOGFILE="/data/myname/projects/rnavar_test/logs/rnavar_run_$(date +%Y%m%d_%H%M%S).log"

nextflow run nf-core/rnavar \
  -profile singularity \
  -resume \
  -c /data/myname/bins/custom.config3 \
  --input /data/myname/projects/rnavar_test/samplesheet_test.csv \
  --outdir /data/myname/projects/rnavar_test/results_test \
  --genome Olea_europaea \
  --skip_baserecalibration \
  --tools snpeff \
  &> "$LOGFILE"
👍 1
✅ 1
ok thank you so much, ill let you know the outcome
it worked! thanks!
Open in Slack
PreviousNext
Powered by