Hi! I am working on implementing pipeline level nf-tests, but I get unstable behaviour: some

nf-test test

runs are successful, but for some I get a failed assertion. I then saw that files that I specify in the .nftignore are still added to the snapshot (which shouldn't happen, right?). The things I double checked is that my .nftignore is in the tests folder (

./tests/.nftignore

), .. I trigger .nftignore in

./nf-test.config

as:

triggers 'nextflow.config', 'nf-test.config', 'conf/test.config', 'tests/nextflow.config', 'tests/.nftignore'

.. and I mention .nftignore my

test_<name>.nf.test

file:

...
then {

            def stable_contents = getAllFilesFromDir(params.outdir,
                                                    relative: false,
                                                    includeDir: false,
                                                    ignoreFile: 'tests/.nftignore'
                                                    ) 
    ...
}

Is there anything else I can check to understand why my

./tests/.nftignore

is not being used when updating the snapshot (I did try to rebuild it several times)? Thank you! 🙏

Do you have an open PR for this?

Unfortunately this is not a public pipeline.

Then to debug, I would suggest making sure the paths are correct

Thanks for the tip! You mean the paths inside

.nftignore

? For example, now, in the

.nftignore

I have **/PCA.png, this regex should exclude PCA.png files in all folders and subfolders, right?

Yeah that should work indeed

Hm, still no luck. I am using "nextflow": "24.04.4" and "nf-test": "0.9.3", could there be any subtle syntax differences?

No there shouldn't. I've been using this since I developed the plugin and no changes in syntax or behavior there yet. Try to print out the list of the files to see more information. And maybe try a more specific path first

Thanks! I tried more specific paths and I tried printing the files to see if my .nftignore is read correctly and it actually does not seem to be the problem. What I did to print the files was to add a print statement inside my

test_<name>.nf.test

:

...
        then {
            def stable_name = getAllFilesFromDir(params.outdir,
                                                relative: true,
                                                includeDir: true,
                                                ignore: ['pipeline_info/*.{html,json,txt}', 'multiqc_dataset/*.html']
                                                )

            def stable_contents = getAllFilesFromDir(params.outdir,
                                                    relative: false,
                                                    includeDir: false,
                                                    ignoreFile: 'tests/.nftignore'
                                                    ) 

            // DEBUGGING
            def all_files = getAllFilesFromDir(
                params.outdir,
                relative: false,
                includeDir: false
            )

            // DEBUGGING 
            println "\n--- IGNORED FILES ---"
            (all_files - stable_contents).each { println it } // <---- printing files that are not in stable_contents, looks good! I see paths to files that I want to exclude.


            assertAll(

                // Check that file names and contents are stable
                { assert snapshot(
                    workflow.trace.succeeded().size(),
                    stable_name,
                    stable_contents
                    ).match() }
....

And yet I get 1 failed assertion

java.lang.RuntimeException: Different Snapshot:

where I see files (that I saw being excluded from stable_contents) having unmatched md5sums ( filename:md5, .1aaa.. | filename:md5, .2bbb.. ) Really puzzled.

Ooooh, nice catch

I'm actually not sure, I think you might be the first one to test this with symlinked files 😁.

That should be fine...

That's what I assumed

It could have something to do with your glob matching? Maybe it doesn't work properly on symlinks

I am now also trying to reproduce the failed assertion on the reduced version of the pipeline. Lets see.