Pieter Moris
02/26/2024, 2:20 PMenv directive and the `APPTAINER_*`/`SINGULARITY_*` environment variables for setting the cache and tmp dir (not the NXF_* one housing the final images, but the apptainer ones).
I've seen a number of example configs (and threads on slack) that use the env directive to set these, e.g.
env {
APPTAINER_TMPDIR="$HOME/.apptainer/tmp"
APPTAINER_CACHEDIR="$HOME/.apptainer/cache"
}
However, in my own testing, these environment variables are not respected. Manually exporting the variables to my shell before launching nextflow does work however.
Digging through the docs, I found [the following](https://www.nextflow.io/docs/latest/config.html#scope-env):
TheSo reading that, am I correct in assuming that the pulling of images and/or converting docker to apptainer, happens in the main nextflow process, prior to any tasks, and thus cannot access any variables defined byscope provides environment variables to tasks, not Nextflow itself. Nextflow environment variables such asenvshould be set in the environment in which Nextflow is launched.NXF_VER
env?Phil Ewels
02/26/2024, 2:30 PMPieter Moris
02/26/2024, 2:32 PMpontus
02/26/2024, 3:06 PMAPPTAINER_TMPDIR is set to set TMPDIR within the container, the correct way of doing that is to set TMPDIR - env scope environments are evaluated within the container context.
So things like that are just plain wrong.Edmund
02/26/2024, 3:08 PMEdmund
02/26/2024, 3:09 PMenvPieter Moris
02/26/2024, 3:10 PMcacheDir actually sets NXF_SINGULARITY_CACHEDIR which indeed does work for me too. But this is only used by nextflow to store the final pulled/built images, not as a cache during pulling/building (which is what SINGULARITY_CACHEDIR and SINGULARITY_TPMDIR do).pontus
02/26/2024, 3:10 PMEdmund
02/26/2024, 3:11 PMPieter Moris
02/26/2024, 3:12 PMwatch and saw what was happening.Pieter Moris
02/26/2024, 3:14 PMPieter Moris
02/26/2024, 3:14 PMpontus
02/26/2024, 3:14 PMpontus
02/26/2024, 3:15 PMEdmund
02/26/2024, 3:17 PMEdmund
02/26/2024, 3:18 PMsingularity.enable...pontus
02/26/2024, 3:19 PMpontus
02/26/2024, 3:20 PMapptainer.enabled, it will yield what nextflow takes as a docker URL, which will be pulled as such.Pieter Moris
02/26/2024, 3:21 PMPieter Moris
02/26/2024, 3:29 PMsingularity.enable), but have apptainer installed on your system (aliased to singularity), pre-built singularity images will be used. It is only when you use the apptainer.enable type profile that docker images will be converted.Edmund
02/26/2024, 6:01 PMpontus
02/26/2024, 6:38 PMPieter Moris
02/27/2024, 8:55 AMenvWhitelist = ['SINGULARITY_TMPDIR'] to allow nextflow to access bashrc variables. However, that doesn't seem to be necessary in my tests. What gives?pontus
02/27/2024, 8:58 AMTMPDIR in env.Pieter Moris
02/27/2024, 9:11 AMSINGULARITY_TMPDIR defines a directory that is used during the build / conversion of singularity images.
⢠It is populated during the master nextflow process, outside of any process tasks.
⢠The env scope only applies to process tasks.
⢠So to make nextflow use it, I though it had to be defined as a global environment variable on whatever system you are running the pipeline.
What's the use case of defining TMPDIR in env then according to you?pontus
02/27/2024, 9:18 AM