hey - thank you very much for developing this pipl...
# bacass
j
hey - thank you very much for developing this pipline! i want (just to try it before i upscale to the cluster) to have the pipeline run locally on two short-read archives. but when i try to do such:
nextflow run nf-core/bacass -r 2.0.0 --input samplesheet.csv -profile docker --skip_kraken2
with a samplesheet like this:
Copy code
ID	R1	R2	LongFastQ	Fast5	GenomeSize
shortreads	./exp_data/file_1.fastq.gz	./exp_data/file_2.fastq.gz	NA	NA	NA
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
it stopps at:
Copy code
[-        ] process > NFCORE_BACASS:BACASS:FASTQC (shortreads)    -
[17/525d28] process > NFCORE_BACASS:BACASS:SKEWER (shortreads)    [100%] 1 of 1, cached: 1 ✔
[-        ] process > NFCORE_BACASS:BACASS:NANOPLOT               -
[-        ] process > NFCORE_BACASS:BACASS:PYCOQC                 -
[70/a51cbd] process > NFCORE_BACASS:BACASS:UNICYCLER (shortreads) [100%] 1 of 1, failed: 1
[-        ] process > NFCORE_BACASS:BACASS:QUAST                  -
[-        ] process > NFCORE_BACASS:BACASS:PROKKA                 -
[-        ] process > NFCORE_BACASS:BACASS:GET_SOFTWARE_VERSIONS  -
[-        ] process > NFCORE_BACASS:BACASS:MULTIQC                -
because i don't have enough cpus on my local machine:
Copy code
Error executing process > 'NFCORE_BACASS:BACASS:UNICYCLER (shortreads)'

Caused by:
  Process requirement exceed available CPUs -- req: 12; avail: 8
this leads me to three questions: • is it possible to alter this requirement? • (i wrote some pipelines myself, but this is the first nf-core pipeline i try to use) where can i access the nextflow config file in a nf core pipeline? • can i extend the samplesheet indefinitely if i want to run the pipeline for many samples like this:
Copy code
ID	R1	R2	LongFastQ	Fast5	GenomeSize
shortreads	./exp_data/sample_1_1.fastq.gz	./exp_data/sample_1_2.fastq.gz	NA	NA	NA
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
shortreads	./exp_data/sample_2_1.fastq.gz	./exp_data/sample_2_2.gz	NA	NA	NA
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
shortreads	./exp_data/sample_3_1.fastq.gz	./exp_data/sample_3_2.gz	NA	NA	NA
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
shortreads	./exp_data/sample_4_1.fastq.gz	./exp_data/sample_4_2.gz	NA	NA	NA
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
.
.
.
thank you very much! best, j.
d
Hi there, here some infos:
• is it possible to alter this requirement?
Sure, you can use
--max_cpus 8 --max_memory '12.GB'
for example. Those are params are hidden in https://nf-co.re/bacass/2.0.0/parameters (which is not optimal imho).
• (i wrote some pipelines myself, but this is the first nf-core pipeline i try to use) where can i access the nextflow config file in a nf core pipeline?
Please do not modify any config of the pipeline itself but provide your own custom config with nextflow's
-c
parameter, see here. However, modify no params in the file used with
-c
but instead use
-params-file,
see here.
• can i extend the samplesheet indefinitely if i want to run the pipeline for many samples like this:
Yes you can extend it indefinitely. But column ID is the sample name and should be unique. All "NA" should be detrimental and lead to an error! I guess that is what you see there. The samplesheet is described in https://nf-co.re/bacass/2.0.0/usage#samplesheet
Those lines:
Copy code
longreads	NA	NA	NA	NA	NA
shortNlong	NA	NA	NA	NA	NA
define two samples named "longreads" and "shortNlong" but define no sequencing files. That makes the pipeline fail. All samples must have at least one sequencing file.
j
thank you very much this helps me a lot!
art those warnings
Copy code
WARN: A process with name 'MINIMAP2_CONSENSUS' is defined more than once in module script: /home/janos/.nextflow/assets/nf-core/bacass/./workflows/bacass.nf -- Make sure to not define the same function as process
WARN: A process with name 'MINIMAP2_POLISH' is defined more than once in module script: /home/janos/.nextflow/assets/nf-core/bacass/./workflows/bacass.nf -- Make sure to not define the same function as process
concerning?
d
no, that was a change in nextflow, an older version of nextflow will not emit that warnings.
because genomeassembler is supposed to be the succossor of bacass, bacass is a little dated in that way, but works fine.
j
excellent - thank you.
another question - if i want to run it via slurm how would i do this?
j
thank you very much - i'll try this! (i have several of my own pipelines running with slurm - but i'am unsure how to extend the existing config file from the bacass pipeline.)
d
So the minimal config to run on slurm for any nextflow to use with
-c
pipeline is
Copy code
process {
  executor = 'slurm'
}
Please do not modify any pipeline file (that might lead to unexpected results, my inability to troubleshoot, and potentially problems when bringing your local copy up-to-date.
j
ah i see - so the additional config exceeds the original one?
d
ah i see - so the additional config exceeds the original one?
yes, the additional config adds info (
process
block) or can even overwrite. So there is no reason whatsoever to modify the pipeline files itself. But please keep in mind that
params
should be used on the command line or in -params-file as detailed above, not via -c. That is, because files with -c are allowed to contain params and those get resolved wrongly when defining params with -c.
j
thank you very much for the explanation!
it worked with the config ... but now i'am stuck with this issue: https://github.com/nf-core/mag/issues/320 do you have by any chance an idea how to resolve this? (i use the singularity profile and tried the different container as mentioned here: https://github.com/nf-core/mag/issues/320#issuecomment-1372921954)
d
Use --skip_annotation to run all that can be done without problem, then follow the issue you linked and add
-resume
to your command that you can test potential solutions without re-running the complete pipeline. If nothing works, and you need the annotation, check out bakta and do it manually.
Or use DFAST which is in the pipeline with https://nf-co.re/bacass/2.0.0/parameters#dfast_config
j
thank you i'll guess i try DFAST (any time horizon when bakta will be included?)
DFAST bites me - since apparently it requires a working internet connection which i do not have on the computing nodes in our cluster.
d
(any time horizon when bakta will be included?)
Will not happen I think, because genomeassembler is supposed to be the successor, work on bacass is minimized.