János Winkler
04/13/2023, 3:03 PMnextflow run nf-core/bacass -r 2.0.0 --input samplesheet.csv -profile docker --skip_kraken2
with a samplesheet like this:
ID R1 R2 LongFastQ Fast5 GenomeSize
shortreads ./exp_data/file_1.fastq.gz ./exp_data/file_2.fastq.gz NA NA NA
longreads NA NA NA NA NA
shortNlong NA NA NA NA NA
it stopps at:
[- ] process > NFCORE_BACASS:BACASS:FASTQC (shortreads) -
[17/525d28] process > NFCORE_BACASS:BACASS:SKEWER (shortreads) [100%] 1 of 1, cached: 1 ✔
[- ] process > NFCORE_BACASS:BACASS:NANOPLOT -
[- ] process > NFCORE_BACASS:BACASS:PYCOQC -
[70/a51cbd] process > NFCORE_BACASS:BACASS:UNICYCLER (shortreads) [100%] 1 of 1, failed: 1
[- ] process > NFCORE_BACASS:BACASS:QUAST -
[- ] process > NFCORE_BACASS:BACASS:PROKKA -
[- ] process > NFCORE_BACASS:BACASS:GET_SOFTWARE_VERSIONS -
[- ] process > NFCORE_BACASS:BACASS:MULTIQC -
because i don't have enough cpus on my local machine:
Error executing process > 'NFCORE_BACASS:BACASS:UNICYCLER (shortreads)'
Caused by:
Process requirement exceed available CPUs -- req: 12; avail: 8
this leads me to three questions:
• is it possible to alter this requirement?
• (i wrote some pipelines myself, but this is the first nf-core pipeline i try to use) where can i access the nextflow config file in a nf core pipeline?
• can i extend the samplesheet indefinitely if i want to run the pipeline for many samples like this:
ID R1 R2 LongFastQ Fast5 GenomeSize
shortreads ./exp_data/sample_1_1.fastq.gz ./exp_data/sample_1_2.fastq.gz NA NA NA
longreads NA NA NA NA NA
shortNlong NA NA NA NA NA
shortreads ./exp_data/sample_2_1.fastq.gz ./exp_data/sample_2_2.gz NA NA NA
longreads NA NA NA NA NA
shortNlong NA NA NA NA NA
shortreads ./exp_data/sample_3_1.fastq.gz ./exp_data/sample_3_2.gz NA NA NA
longreads NA NA NA NA NA
shortNlong NA NA NA NA NA
shortreads ./exp_data/sample_4_1.fastq.gz ./exp_data/sample_4_2.gz NA NA NA
longreads NA NA NA NA NA
shortNlong NA NA NA NA NA
.
.
.
thank you very much!
best,
j.Daniel Straub
04/14/2023, 7:45 AM• is it possible to alter this requirement?Sure, you can use
--max_cpus 8 --max_memory '12.GB' for example. Those are params are hidden in https://nf-co.re/bacass/2.0.0/parameters (which is not optimal imho).
• (i wrote some pipelines myself, but this is the first nf-core pipeline i try to use) where can i access the nextflow config file in a nf core pipeline?Please do not modify any config of the pipeline itself but provide your own custom config with nextflow's
-c parameter, see here. However, modify no params in the file used with -c but instead use -params-file, see here.
• can i extend the samplesheet indefinitely if i want to run the pipeline for many samples like this:Yes you can extend it indefinitely. But column ID is the sample name and should be unique. All "NA" should be detrimental and lead to an error! I guess that is what you see there. The samplesheet is described in https://nf-co.re/bacass/2.0.0/usage#samplesheet
Daniel Straub
04/14/2023, 7:48 AMlongreads NA NA NA NA NA
shortNlong NA NA NA NA NA
define two samples named "longreads" and "shortNlong" but define no sequencing files. That makes the pipeline fail. All samples must have at least one sequencing file.János Winkler
04/14/2023, 7:53 AMJános Winkler
04/14/2023, 8:05 AMWARN: A process with name 'MINIMAP2_CONSENSUS' is defined more than once in module script: /home/janos/.nextflow/assets/nf-core/bacass/./workflows/bacass.nf -- Make sure to not define the same function as process
WARN: A process with name 'MINIMAP2_POLISH' is defined more than once in module script: /home/janos/.nextflow/assets/nf-core/bacass/./workflows/bacass.nf -- Make sure to not define the same function as process
concerning?Daniel Straub
04/14/2023, 8:06 AMDaniel Straub
04/14/2023, 8:07 AMJános Winkler
04/14/2023, 8:16 AMJános Winkler
04/18/2023, 1:30 PMDaniel Straub
04/18/2023, 2:30 PMJános Winkler
04/19/2023, 6:21 AMDaniel Straub
04/19/2023, 7:08 AM-c pipeline is
process {
executor = 'slurm'
}
Please do not modify any pipeline file (that might lead to unexpected results, my inability to troubleshoot, and potentially problems when bringing your local copy up-to-date.János Winkler
04/19/2023, 8:05 AMDaniel Straub
04/19/2023, 10:22 AMah i see - so the additional config exceeds the original one?yes, the additional config adds info (
process block) or can even overwrite. So there is no reason whatsoever to modify the pipeline files itself. But please keep in mind that params should be used on the command line or in -params-file as detailed above, not via -c. That is, because files with -c are allowed to contain params and those get resolved wrongly when defining params with -c.János Winkler
04/19/2023, 10:51 AMJános Winkler
04/19/2023, 11:48 AMDaniel Straub
04/19/2023, 12:20 PM-resume to your command that you can test potential solutions without re-running the complete pipeline. If nothing works, and you need the annotation, check out bakta and do it manually.Daniel Straub
04/19/2023, 12:21 PMJános Winkler
04/19/2023, 12:24 PMJános Winkler
04/19/2023, 2:44 PMDaniel Straub
04/20/2023, 8:03 AM(any time horizon when bakta will be included?)Will not happen I think, because genomeassembler is supposed to be the successor, work on bacass is minimized.