Hello! I am trying to run the nf-core/ampliseq pipeline on approx 2300 samples, but I am new to ampliseq and I never get past the

DADA2_DENOISING

step. After 3-5 days the process is

Killed

. How can I debug, and check if I need to change something in the setup? Is it ok to run this from login node? Any advice on how to troubleshoot would be very welcome.

nextflow run nf-core/ampliseq \

-r 2.12.0 \

-profile pdc_kth,singularity \

-resume \

--project naiss2025-22-41 \

--input samples_list_clean_sane.tsv \

--outdir results \

-work-dir work \

--skip_cutadapt \

--skip_qiime \

--skip_taxonomy \

--skip_dada_taxonomy \

--skip_dada_addspecies \

--ignore_failed_trimming \

--ignore_empty_input_files \

--ignore_failed_filtering

We have a channel for #CEA7TBJGJ. Direct your question there next time.

Killed

means you ran out of memory. You have an incredible number of samples which might make it difficult to run Ampliseq in one go for all. I typically run the pipeline once for every sequencing round.

Ok thank you! Should I move this question now also? This is one sequencing project 😅 So I will most likely need to split it up? What is a reasonable max number of samples?

Right. We sequence 96 samples at a time. I didn't know one could sequence 2300 samples in one go. What did you set

--sample_inference

to? If you set it to

independent

(the default) denoising of each sample should be independent of the others and running the pipeline a couple of times with a subset of samples should not effect the results. No need to move this. But when you have a follow up post in #CEA7TBJGJ.

I did not set

--sample_inference

, so it will have been default? Is say 500-600 samples still quite a lot as a subset, or more reasonable?

Yes, then you used the default. I would start by trying say a quarter or a fifth (i.e. 400-600 like you suggest) in each. Run that and see if you get it through. The larger the batches, the easier to keep control of. An alternative would be to allocate more memory to the process but that might be more difficult to get through the queue if you're running on an HPC. Here's a general intro to how you control execution options like memory with config files: https://nf-co.re/docs/usage/getting_started/configuration#custom-configuration-files. We can help with details later if you need that.

Ok - thanks a lot for the advice!

Another thing to think about: The taxonomy assignment with DADA2 is not completely reproducible, and the same ASV can get different taxonomy assignments. (I typically get 2-3 differences between two runs having 5-10000 ASVs.) After denoising of all batches finishes, you might hence want to rerun taxonomy using a fasta file with all unique ASVs, see https://nf-co.re/ampliseq/2.15.0/docs/usage/#asvotu-fasta-input.

Ok! I am using

--skip_taxonomy

so this will be skipped, or?

Correct