Another question: It fails towards the end where I...
# helpdesk-hpc-sweden
k
Another question: It fails towards the end where I get an error that the disk quota is exceeded and I think it's about the singularity cache
Should I specify the cachedir in some way so it's not using from my user quota (it looks like it's by default put there) but from the storage project quota
+ should I then both set the NXF_SINGULARITY_CACHEDIR and the NXF_APPTAINER_CACHEDIR or only the former?
p
Yes, that's probably helpful.
It doesn't hurt to set both, but
NXF_SINGULARITY_CACHEDIR
should be the important one (or having a
nextflow.config
that does the corresponding thing).
k
Something like this:
export NXF_APPTAINER_CACHEDIR=/cfs/klemming/projects/snic/uppstore2017270/common_tools/NF-CORE_PANGENOME_APPTAINER_CACHEDIR
mkdir -p $NXF_APPTAINER_CACHEDIR
export NXF_SINGULARITY_CACHEDIR=/cfs/klemming/projects/snic/uppstore2017270/common_tools/NF-CORE_PANGENOME_SINGULARITY_CACHEDIR
mkdir -p $NXF_SINGULARITY_CACHEDIR
p
Looks good. (The profile is currently set to use the
singularity
container engine and thus should pick up that option, but for the jobs, we're loading the
apptainer
module and relying on that to provide the
singularity
binary 🙂
k
Any reason it's the apptainer module and not the singularity one?
p
Yes, it's due to how PDC had built the tooling (at least at some earlier point in time).
k
ok!
Can I safely remove the
.singularity
and
.apptainer
directories in my home directory that currently takes up quite some space?
p
The singularity provided would unpack the container images whereas the apptainer setup would mount them (only using what's needed).
Yes, they're fine to remove. They will be recreated though.
k
Yeah, but with less things in them if I remember to set the cache dirs elsewhere from now, right
👍 1
Ok, testing again. It almost reached the end yesterday, it failed on the last multiqc step where it's collecting all stats 🤦