Hi all, Trying to write an nf-test for my module. I get an error that I'm not sure what to do with. It says all my tests passed and succeeded, but then it says that nf-test failed. I changed my test assertions to just a simple process.sucess and version check, and they do indeed pass, and the --verbose output also does not really give a hint as to what's going wrong. Anyone have encountered something similar?

Try running nf-test test path/to/main.nf.test --profile docker, and see if that gives a more informative error

nf-test test modules/nf-core/nasvar/pipeline/tests/main.nf.test --profile docker --verbose

🚀 nf-test 0.9.5
<https://www.nf-test.com>
Please cite: <https://doi.org/10.1093/gigascience/giaf130>
(c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr

Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar
Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar
Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar
Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar
Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar
Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar
Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar

Test Process NASVAR_PIPELINE

  Test [f427fc01] 'nasvar pipeline - bam - RUNX1::ETV6'
    > Nextflow 26.04.3 is available - Please consider updating your version to it
    > N E X T F L O W  ~  version 26.04.0
    > Launching `/Users/k.v.cammel/Documents/Projects/Leukemia/nasvar_docker/nasvar_nextflow/modules/.<http://nf-test-f427fc01d6706a560b6779545a496080.nf|nf-test-f427fc01d6706a560b6779545a496080.nf>` [golden_shaw] - revision: 69d907beb3
    > [bf/77024e] Submitted process > NASVAR_PIPELINE (test)
    PASSED (135.392s)
  Test [e96eaa29] 'sarscov2 - bam - stub'
    > Nextflow 26.04.3 is available - Please consider updating your version to it
    > N E X T F L O W  ~  version 26.04.0
    > Launching `/Users/k.v.cammel/Documents/Projects/Leukemia/nasvar_docker/nasvar_nextflow/modules/.<http://nf-test-e96eaa2919482a0a0796b00e773fdbf9.nf|nf-test-e96eaa2919482a0a0796b00e773fdbf9.nf>` [silly_bohr] - revision: c19d145531
    > [91/aa42e7] Submitted process > NASVAR_PIPELINE (test)
    PASSED (3.297s)


SUCCESS: Executed 2 tests in 138.755s

Well, nice that it passed. But then could it be an issue with the nf-core implementation of nf-test?

it shoudl just be a wrapper. What does nf-core modules test give?

```
Then I get those kind of strange:
```CRITICAL Ran, but found errors:
          - nf-test failed
          - nf-test failed

So, it looks very much like all the tests pass, but then it just ends with these mysterious errors

Hmm, something is not right then. Can you update your nf-core tools?

I did a bit of digging and I think the problem might be with

<https://github.com/nf-core/tools/blob/main/nf_core/components/components_test.py#L191>

def display_nftest_output(self, nftest_out: bytes, nftest_err: bytes) -> None:
        if self.no_prompts:
            # In no_prompts mode (e.g., tests), use simple logging to avoid Rich hanging in non-TTY environments
            log.debug("nf-test output:\n%s", nftest_out.decode())
            if nftest_err:
                log.debug("nf-test error:\n%s", nftest_err.decode())
            if "Different Snapshot:" in nftest_err.decode() and self.update:
                <http://log.info|log.info>("Updating snapshot")
                self.generate_snapshot()
        else:
            # Interactive mode: use Rich formatting
            print("Displaying nf-test output")
            nftest_output = Text.from_ansi(nftest_out.decode())
            print(Panel(nftest_output, title="nf-test output"))
            print("Displaying nf-test error", nftest_err.decode())
            if nftest_err:
                print("Parsing nf-test error")
                syntax = Syntax(nftest_err.decode(), "diff", theme="ansi_dark")
                print("Parsed nf-test error")
                print(Panel(syntax, title="nf-test error"))
                print("Displaying nf-test error")
            if "Different Snapshot:" in nftest_err.decode():
                log.error("nf-test failed due to differences in the snapshots")
                if self.update is None:
                    answer = Confirm.ask(
                        "[bold][blue]?[/] nf-test found differences in the snapshot. Do you want to update it?",
                        default=True,
                    )
                    if answer:
                        <http://log.info|log.info>("Updating snapshot")
                        self.update = True
                    else:
                        log.debug("Snapshot not updated")
                if self.update:
                    # update snapshot using nf-test --update-snapshot
                    self.generate_snapshot()

            else:
                self.errors.append("nf-test failed")

So in "interactive mode", it checks if nftest_err exists, then it checks for "Different Snapshot" in nftest_err, but if it does not find that it still prints (nf-test failed) even if nftest_err is empty. If I run with

nf-core modules test nasvar/pipeline --profile docker --verbose --no-prompts

I don't get these errors

nf-core modules test nasvar/pipeline --profile docker --verbose --no-prompts



                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 4.0.2 - <https://nf-co.re>


INFO     Generating nf-test snapshot
INFO     Generating nf-test snapshot again to check stability
INFO     All tests passed!

Can you try the development version before you do: https://nf-co.re/docs/nf-core-tools/cli/installation#development-version

I still get the same error

Make an issue on the nf-core/tools repository then