Can someone explain to me `PYTHONNOUSERSITE`? I am...
# nostupidquestions
n
Can someone explain to me
PYTHONNOUSERSITE
? I am trying to make a container for a python package called
mifaser
. I don’t have any issues running the container directly with docker or podman:
Copy code
<docker/podman> run  <http://ghcr.io/vdblab/mifaser:1.64b|ghcr.io/vdblab/mifaser:1.64b> mifaser --help
However, when running via nextflow with podman, I encounter the error:
Copy code
Command error: Traceback (most recent call last):
      File "/home/mambauser/.local/bin/mifaser", line 5, in <module>
        from mifaser.__main__ import main
    ModuleNotFoundError: No module named 'mifaser'
When I try via nextflow with docker, I get
Copy code
.command.sh: line 4: mifaser: command not found
If I force
PYTHONNOUSERSITE="0"
in the script body, the process runs fine with nextflow+podman! But, it still fails with
.command.sh: line 2: mifaser: command not found
via nextflow+docker. Any tips would be appreciated! Dockerfile here: https://raw.githubusercontent.com/vdblab/dockerfiles/refs/heads/main/mifaser/Dockerfile
m
Nextflow forces
PYTHONNOUSERSITE=1
when running jobs. This is to ensure user installs don’t override the container. If your pipeline / tool requires
PYTHONNOUSERSITE="0"
to run, that means the software isn’t fully packaged in the container !
A container should really contain all dependencies. Here I wonder if the problem is that you
pip3 install
the software as
mambauser
, which makes the software go under
/home/mambauser/.local/bin
, and even though you add it to
PATH
,
PATH
is not considered during the installation. I think I’ve seen other cases where the effective
PATH
when running the pipeline is something like
/usr/local/bin:/usr/bin:/sbin:/bin
I would suggest moving
USER mambauser
to the very end of the file so that the software is installed in
/usr/local/bin
. You wouldn’t need to add
/home/mambauser/.local/bin
to
PATH
.
I’m not sure the tool would see diamond, though 🤔
An alternative to rebuilding your container would be to add this to the top of your script body in Nextflow
Copy code
export PATH="/opt/conda/bin:/home/mambauser/.local/bin:${PATH}"
n
Thanks, this is great info! I gotta start testing these containers with
PYTHONNOUSERSITE=1
. I naively thought that the anything lacking dependencies would just fail by default rather than looking in the executing env.
p
PYTHONNOUSERSITE
is also required/useful for e.g. singularity where it helps making sure that you don't accidentally pick up a dependency that's installed in the user directory. But that shouldn't affect whatever a tool shows up in
PATH
or not.