I'm trying to test a workflow that needs as input a nextclade dataset name (eg

sars-cov-2

). In the

test

file I have a this setup block

setup {
        run("NEXTCLADE_DATASETGET") {
            script "../modules/nf-core/nextclade/datasetget/main.nf"
            process {
                """
                input[0] = 'nextstrain/sars-cov-2/wuhan-hu-1/orfs'
                input[1] = '2024-01-16--20-31-02Z'

                """
            }
        }
    }

But I'm unsure how I can back reference this in

conf/test.config

. How do I test a pipeline that requires the above?

It’s unclear to me what or why you need to back reference in the config?

To get the pipeline to run nextclade arm,

params.nextclade

must be a path or a string. This needs to be specified in

conf/test.config

I think I am still missing something. If nextclade runs as part of the pipeline, then why is it being run in the setup? Is it just to pre-download the reference data? If so then I don’t know that this really works for a pipeline level test. AFAIK the

setup

is intended to help for modules/subworkflows that are designed to take input that is typically chained from another module, not to give some method to pre-download external dependencies outside of a pipeline under test. If that’s the case then I think what you are trying to achieve is essentially:

when { 
   params {
       nextclade = NETXCLADE_DATASET.out.dataset.value

But of course this would never work.

I think I am still missing something. If nextclade runs as part of the pipeline, then why is it being run in the setup? Is it just to pre-download the reference data?

It's not run as part of the standard pipeline, the user has to specify it.

But then specifying it will cause it to run, so no need to run the setup step?

It's just that

sars-cov-2

is actually telling nextclade to download the latest, which changes frequently. I guess the output itself is not that important, only that the output exists.

So from what you say it sounds like that even if the setup step ran, if the

params.nextclade

was set it would run again in the pipeline anyway as that just controls if it runs or not, and there isn’t actually a param to pass in a pre-downloaded dataset?

Yeah that's right,

setup

block becomes redundant if

params.nextclade

is specified.

But if it’s not set then what does

setup

do? Because surely the actual

nextclade

never runs if the param is unset

For that test, I always want

nextclade

to run.

👍

Thanks again for your help Arthur.

So I guess you have a few options: • accept that when testing with it enabled the test will use the latest verison of sars-cov-2 it can download • expose some way to configure the

tag

input of your pipeline’s implementation of

nextclade_datasetget

and use that to fix the version pulled in for the tests (exposing it is probably good anyway I would have thought) • expose a way to pass in a pre-downloaded dataset, which you would still need to fetch for your tests some other way than

setup

Even if you do have a way to pin the version of the dataset that is used in tests it might still be good to test the latest version anyway (perhaps without checking the md5sum of relevant outputs stay the same) because if that’s mostly how the pipeline is run, then you want to check it’s not broken