In terms of implementations, I see a couple of options • implement manual tests on the results (essentially duplicating some functionality of nf-test) • implement a "fake" nf-test pipeline that just fetches the results from the seqera platform run, but then uses nf-test's functionality to check assertions • somehow modify nf-test such that it supports seqera platform as a backend to submit the pipeline runs. Maybe this is already possible, by adding the corresponding config options to nextflow. Just the pipeline wouldn't be triggered via seqera platform API, but it would be a local nextflow run that can be monitored on seqera platform.

And that's definitively what we should be doing at some point

My guess is with current setup it's going to be hard to do

I would actually just want to uncouple the execution from the tests, and be able to verify the output

basically in my mind, what we're currently using nf-test for in the pipeline for pipeline tests is doing unit test at a global level, when we only check the output

I'm progressing towards a test declaration as simple as possible for pipeline level tests

and at some point just checking out the content towards a snapshot would be the endgoal

Probably something like pytest-workflow could do that already (which we have been using in the pre-nf-test era)

I would say that all depends on what you actually want to check, if it's file name or existence, it's easy

and a good start already

I think I will start playing with that idea more

Thanks @Gregor Sturm for the inspiration

love the idea. Definitely long overdue

ok, a really dumb way of doing that would be to implement a nextflow process that runs

aws s3 cp <pipeline result> .

and test that with nf-test. Should work in principle, no? If such a prototype is successful, we could request the nf-test team to add a new directive for testing folders without running nextflow code, such as

folder { 
  path "./s3_download"
  test(
     assert path("output.fastq.gz").md5 == "4a17df7a54b41a84df492da3f1bab1e3"
  )
}