<#775 disk quota exceeded in directory ~/.singular...
# configs
g
#775 disk quota exceeded in directory ~/.singularity/cache/net when using NCI Gadi HPC Configuration Issue created by mathob I am trying a workflow using the nci_gadi profile but am encountering a disk quota problem at the stage of pulling singularity images. At this point nextflow wants to use the directory ~/.singularity/cache for caching but my home directory filesystem is subject to a stringent disk space restriction. Setting the environment variable NXF_SINGULARITY_CACHEDIR to a directory under /scratch (not subject to the same space constraint) works insofar as images end up being saved there; however the singularity pull commands continue to use the directory ~/.singularity/cache/net to save temporary downloads which accumulate there until eventually I exceed my disk quota. So my question is: how do I configure nextflow to use a directory other than ~/.singularity/cache/net for singularity pull commands? I had thought that environment variable SINGULARITY_TMPDIR (or maybe NXF_SINGULARITY_TMPDIR?) would be relevant but I have not figured out how make nextflow respect them. My command is:
nextflow run nf-core/rnaseq --input SampleSheet.csv --outdir ${PWD}/results --gtf ${PWD}/mm10/gencode.vM10.primary_assembly.annotation.gtf --fasta ${PWD}/mm10/GRCm38.primary_assembly.genome.fa --aligner star_rsem -profile nci_gadi,singularity
Some log info is attached (username and project details obfuscated) log.txt The actual error message is:
Copy code
Failed to pull singularity image
    command: singularity pull  --name depot.galaxyproject.org-singularity-fastqc-0.12.1--hdfd78af_0.img.pulling.1728977625666 <https://depot.galaxyproject.org/singularity/fastqc:0.12.1--hdfd78af_0> > /dev/null
    status : 255
    hint   : Try and increase singularity.pullTimeout in the config (current is "20m")
    message:
      INFO:    Downloading network image
      INFO:    Cleaning up incomplete download: /home/xxx/[myuser]/.singularity/cache/net/tmp_4260206683
      FATAL:   write /home/xxx/[myuser]/.singularity/cache/net/tmp_4260206683: disk quota exceeded
The ~/.singularity/cache/net directory fills up with files like:
Copy code
25M 87cf6042ef882b1562d5793dad2386a6256d3b7a4210d05002fa5b9133989fbf
 61M 9bcc502e675ace760d757418328c6526cd8afac7ea028e6cd2ebd99e73aa0ff9
168M tmp_215370473
128M tmp_3968314656
Thanks for any suggestions! Matthew nf-core/configs