Hi, I'm trying to add single-m to the taxprofiler pipeline. For now, I'm just trying to get it to run locally before submitting a PR. I've followed the directory setup the way the rest of the repo is set up. This is the relevant part of the config in params:

// singlem
    run_singlem                                          = true
    singlem_save_reads                                   = false
    singlem_save_log                                     = false

This is the relevant parts of the subworkflow in profiler.nf:

include { SINGLEM_PIPE                                  } from '../../modules/nf-core/singlem/singlem/main'
(...)
if (params.run_singlem) {
    ch_input_for_singlem = reads
        .map { meta, input_reads -> tuple(meta, input_reads) }

    SINGLEM_PIPE(ch_input_for_singlem)

    // Collect outputs
    ch_raw_profiles   = ch_raw_profiles.mix(SINGLEM_PIPE.out.results)
    ch_versions       = ch_versions.mix(SINGLEM_PIPE.out.versions)
}

Note SingleM does not initially a db to run. This is the process:

process SINGLEM_PIPE {
    tag "$meta.id"
    label 'process_high'

    conda "${moduleDir}/environment.yml"
    container "${ workflow.containerEngine == 'singularity' ?
        '<docker://wwood/singlem:0.20.3>' :
        '<docker://wwood/singlem:0.20.3>' }"

    input:
    tuple val(meta), path(reads)

    output:
    tuple val(meta), path('*.profile.tsv'), emit: results
    path "versions.yml", emit: versions

    when:
    task.ext.when == null || task.ext.when

    script:
    def args   = task.ext.args ?: ''
    def prefix = task.ext.prefix ?: "${meta.id}"
    def input_args = meta.single_end ? "-1 ${reads[0]}" : "-1 ${reads[0]} -2 ${reads[1]}"

    """
    singlem data
    export SINGLEM_METAPACKAGE_PATH=~/.singlem
    singlem pipe \\
        $input_args \\
        -p ${prefix}.profile.tsv \\
        $args

    cat <<-END_VERSIONS > versions.yml
    "${task.process}":
        singlem: \$(singlem --version 2>&1)
    END_VERSIONS
    """
}

This is the environment.yml:

---
# yaml-language-server: $schema=<https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json>
channels:
  - conda-forge
  - bioconda
dependencies:
  - bioconda::singlem>=0.20.3

This is the link to the SingleM documentation: https://wwood.github.io/singlem/tools/pipe https://wwood.github.io/singlem/Installation I've made a sample sheet in the base directory that looks like this:

sample,run_accession,instrument_platform,fastq_1,fastq_2,fasta
2611,ERR5766174,ILLUMINA,,,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fasta/ERX5474930_ERR5766174_1.fa.gz>
2612,ERR5766176,ILLUMINA,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474932_ERR5766176_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474932_ERR5766176_2.fastq.gz>,
2612,ERR5766176_B,ILLUMINA,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474932_ERR5766176_B_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474932_ERR5766176_B_2.fastq.gz>,
2612,ERR5766180,ILLUMINA,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474936_ERR5766180_1.fastq.gz>,,
2613,ERR5766181,ILLUMINA,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474937_ERR5766181_1.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERX5474937_ERR5766181_2.fastq.gz>,
ERR3201952,ERR3201952,OXFORD_NANOPORE,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fastq/ERR3201952.fastq.gz>,,
2614,ERR3201952,OXFORD_NANOPORE,,,<https://raw.githubusercontent.com/nf-core/test-datasets/taxprofiler/data/fasta/ERR3201952_dup.fa.gz>

And an empty database sheet in the base directory that looks like this:

tool,db_name,db_params,db_path

When I run with the command:

nextflow run <http://main.nf|main.nf>  -profile conda  --input samplesheet.csv --databases databases.csv --outdir test_out

I get this error:

raise Exception("The metapackage defined by the {} environment variable is either malformed or does not match the version encoded in the version of SingleM installed ({}). If you are wanting to run a custom metapackage (or one newer than the installed software) then use the --metapackage flag, rather than specifying the metapackage through the {} environment variable.".format(
  Exception: The metapackage defined by the SINGLEM_METAPACKAGE_PATH environment variable is either malformed or does not match the version encoded in the version of SingleM installed (5.4.0). If you are wanting to run a custom metapackage (or one newer than the installed software) then use the --metapackage flag, rather than specifying the metapackage through the SINGLEM_METAPACKAGE_PATH environment variable.

I'm not sure how to get around this error.

I don't understand the tool - it's not my area of expertise, but looking at the error message, and throwing this at AI, it gave some plausible answers (all completely unverified). Based on the error and the SingleM documentation, the issue is that:

singlem data

downloads the latest metapackage to

~/.singlem

The downloaded metapackage version (5.4.0) doesn't match what SingleM expects 1. SingleM suggests using the

--metapackage

flag instead of the environment variable Here's the fix for your process: Solution 1: Let SingleM download the metapackage on-the-fly (Recommended) Modify your process to let SingleM download and use the metapackage automatically without setting the environment variable:

process SINGLEM_PIPE {
    tag "$meta.id"
    label 'process_high'

    conda "${moduleDir}/environment.yml"
    container "${ workflow.containerEngine == 'singularity' ?
        '<docker://wwood/singlem:0.20.3>' :
        '<docker://wwood/singlem:0.20.3>' }"

    input:
    tuple val(meta), path(reads)

    output:
    tuple val(meta), path('*.profile.tsv'), emit: results
    path "versions.yml", emit: versions

    when:
    task.ext.when == null || task.ext.when

    script:
    def args   = task.ext.args ?: ''
    def prefix = task.ext.prefix ?: "${meta.id}"
    def input_args = meta.single_end ? "-1 ${reads[0]}" : "-1 ${reads[0]} -2 ${reads[1]}"

    """
    singlem pipe \\
        $input_args \\
        -p ${prefix}.profile.tsv \\
        $args

    cat <<-END_VERSIONS > versions.yml
    "${task.process}":
        singlem: \$(singlem --version 2>&1)
    END_VERSIONS
    """
}

The key change: Remove the

singlem data

and

export SINGLEM_METAPACKAGE_PATH

lines. SingleM will automatically download the appropriate metapackage when it runs for the first time.

@Dennis Godin I think as a first step it would make sense to go into the work directory and see if you can get it to work there.... I have a feeling this is a tool specific issue. Try editing the

.command.sh

to print stuff to see if it finds stuff correctly I'm wondering if the

~/.singlem

doesn't exist on your machine - but even if you don't use it, it still needs to exist?

Another solution was to pre-download metapackage to a persistent location, which might be good to download once and use it as a requirement if it's a large file?

This is what we want to do, and the location need sto be specified in the

database.csv