Hi all, has anyone here ran into caching issues running

scnanoseq

(in an HPC environment)? After multiple attempts, nothing is ever cached, so it's not just one specific process that is getting mixed. Wondering if it might be on my end. Code example:

#SBATCH stuff

export NXF_WORK="$VSC_SCRATCH/work"

nextflow run nf-core/scnanoseq \
	--input /some/path/scnanoseq/samplesheet.csv \
	--outdir /some/path/scnanoseq/results \
	--fasta /some/path/fasta/genome.fa \
	--gtf /some/path//genes/genes.gtf \
	--whitelist /some/path//cellranger_arc_rna.737K-arc-v1.txt.gz \
	--barcode_format 10X_3v3 \
	--split_amount 500000 \
	-c /some/path/scnanoseq/custom_scnanoseq.conf \
	-profile vsc_kul_uhasselt,wice,singularity \
	-resume

Hi Luuk - so a while back I did get an issue with caching (https://github.com/nf-core/scnanoseq/issues/8), but I didn’t get good reproducibility on the issue and eventually went away without us finding the root cause of it. That being said I never come across a case where nothing is caching…, Is this also happening if you run scnanoseq with the

test

profile?

Hi Lara, Just tested it with the

-profile test

, making sure everything else is the same (same profiles for cluster submission and workdir, etc.). After running it twice, everything was cached as intended on the second run. So this seems to be fine.

Got it - as I said, it is an odd behavior that we have sporadically noted in the past. But without reproducing it consistently it will be difficult to debug. @Austyn Trull what do you think. We could bring this up to the core team (perhaps during their “office hours”).

I hopefully have some more time as well next week. I'll do some digging as well. I've noticed this in the past with some pipelines I've made as well, but mostly this was just one step that wouldn't cache (and thus future steps would rerun as well). I've never seen it from the first step of the workflow.

Yeah could be worth it, only real thoughts is it being related to some of the splitting/merging. But wouldn’t explain it going all the way to the start of the pipeline. May be worth doing some additional testing on our end as well to see if we can ‘force’ it to happen again