I'm trying to resolve a merge conflict with my PR ☝️ and the current

dev

but get this error from the

DOWNLOAD_REFERENCE_QIIME

module:

Command executed:
  
    wget -O "85_otus.fasta" "<https://data.qiime2.org/2023.7/tutorials/training-feature-classifiers/85_otus.fasta>"
  
  Command exit status:
    4
  
  Command output:
    (empty)
  
  Command error:
    --2026-03-17 15:18:44--  <https://data.qiime2.org/2023.7/tutorials/training-feature-classifiers/85_otus.fasta>
    Resolving <http://data.qiime2.org|data.qiime2.org> (<http://data.qiime2.org|data.qiime2.org>)... failed: Temporary failure in name resolution.
    wget: unable to resolve host address '<http://data.qiime2.org|data.qiime2.org>'

This happens both with

nf-test

and running the test case directly through Nextflow. The url resolves fine in the browser and my command line. We've had similar issues with downloads from figshare that were resolved by turning to an alternative url. I get the same error with the

doubleprimers

test, but now with Greengenes:

Command executed:
  
    wget -O "16S_Greengenes13.5_20200326.tgz" "<https://genome-idx.s3.amazonaws.com/kraken/16S_Greengenes13.5_20200326.tgz>"
  
  Command exit status:
    4
  
  Command output:
    (empty)
  
  Command error:
    --2026-03-17 15:31:43--  <https://genome-idx.s3.amazonaws.com/kraken/16S_Greengenes13.5_20200326.tgz>
    Resolving <http://genome-idx.s3.amazonaws.com|genome-idx.s3.amazonaws.com> (<http://genome-idx.s3.amazonaws.com|genome-idx.s3.amazonaws.com>)... failed: Temporary failure in name resolution.
    wget: unable to resolve host address '<http://genome-idx.s3.amazonaws.com|genome-idx.s3.amazonaws.com>'

Sorry that I cause problems for your PR. I tested my changes with apptainer (new versions of singularity under a new name) and the CI tests with docker also didnt break. I didnt see any issue, I basically used a slightly modified version of the nf-core module https://nf-co.re/modules/wget/. Is there something specific I can try/investigate to help?

As I mentioned, this happens with that module also in other pipelines every now and then. For the time being I'm using singularity so it works. However, I had another problem with the merge. The sintax test failed with the below message which doesn't seem to have anything to do with phyloplacement. Now, I got it through, but this is quite troubling...

Command error:
    INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
    INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
    QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
    Plugin error from alignment:
    
      [Errno 2] No such file or directory: '/tmp/qiime2/daluab/processes/41-1773773093.32@daluab/tmp/qiime2-provenance-7ypkqcun/citations.bib'

As I mentioned, this happens with that module also in other pipelines every now and then. For the time being I'm using singularity so it works.

Ok, so that means the wget module is sort of imperfect? I mean I can also make the pipeline omit the download process when no reference storage dir is specified, i.e. that it would work as before in that case. Do you think that would be of advantge?

The sintax test failed with the below message which doesn't seem to have anything to do with phyloplacement. Now, I got it through, but this is quite troubling...

Unfortunately I see that

[Errno 2] No such file or directory:

for QIIME2 from time to time. I think this is a problem of the tmp handling of QIIME2 in general. I attempted to make that more reliable but I wasnt able to (maybe lack of understanding that issue). Fortunately, that problem seems to me more prevalent with CI tests than with actual runs on our HPC.

> As I mentioned, this happens with that module also in other pipelines every now and then. For the time being I'm using singularity so it works.

Ok, so that means the wget module is sort of imperfect? I mean I can also make the pipeline omit the download process when no reference storage dir is specified, i.e. that it would work as before in that case. Do you think that would be of advantge?

If one can circumvent calling that module, I think the pipeline would be less prone to problems. Nextflow staging is probably better. In some cases there's no alternative though. (But, now that I think about it, it was probably staging that caused problems with the sbdi-gtdb files when I used the other urls.)

Now I am uncertain, the conclusion is still to avoid wget module if possible, right?

Yes, I'd say so, staging seems more stable. Still, in certain cases, using wget is better even if there's a price to pay in the form of some stability problems.

https://github.com/nf-core/ampliseq/pull/968 🙂