A community effort to collect a curated set of analysis pipelines built using Nextflow.

nf-core

Can we swap the github CI/CD for modules to use the `nf-test --only-changed` <https://www.nf-test.com/docs/cli/test/|option>?
I know that <@UV41DBFAT> has been using this in nascent: <https://github.com/nf-core/nascent/blob/dev/.github/workflows/ci.yml>

<@U02AYM7HVR7> you tested this approach I think?

Think we can deprecate pytest-workflow and block any PRs to modules until they’re converted?

I think it will vastly simplify the CI

We are about ~60% complete on the conversion

<@U04M8EZHEFL> has really helped with a script to automate the simple cases

I think we are probably at the point of being able to deprecate pytest

but it would require an extra push to help get the remaining modules swapped over...

I meant by deprecate we still have the tests, we just make it mandatory to make any changes to them, they need to be swapped to nf-test.

By now we should've converted the majority of the popular modules.

We can add a tag like "Needs nf-test help" or something and some of the maintainers can pick up the conversions if the contribution is small or the contributor doesn't have an interest in helping.

I ask people to swap to nf-test and say that I'm willing to help (and do it if they can't/won't), so that is no different to now :wink:

There are still 316 pytest modules with one include statement which can be converted automatically. Then, there are 196 cases where chaining is needed. My hope is that most of the chained test can be converted with regex. There will be many edge cases along the way. They can be taken care of. Conda fails will be a headache, though.

Conda fails are always a headache :slightly_frowning_face:

We need some way to document them, and just move on :thinking_face:

<https://github.com/bioconda/bioconda-recipes/blob/master/recipes/rsem/build_failure.osx-64.yaml>

That is a good idea <@UV41DBFAT>
Maybe we can setup the CI to save a conda-fail.yml file with the error log in the module’s directory if only the conda profile fails while the docker and singularity profiles are successful. The reviewers can look at the error, the current version and the latest version in the fail.yml to see if the fail can’t be fixed at the moment. The CI will ignore conda tests for modules with a fail.yml.

<https://github.com/nf-core/modules/pull/6286/commits/651561a6977359031853d86ad7552dcc4eeda503>
Something like this I'm thinking

Good start. If we are restructuring the CI, I suggest that we also get rid of the conda skip list. Rather we check if the `environment.yml` file is missing or not. If missing, the conda test passes automatically. This will also simplify the conda fail logic.

Love that idea. 

But I’m not sure if either of them will work with the sharding. 

Great! We do have `adamrtalbot/detect-nf-test-changes`  as a fallback option?

That's what we're currently using, so I don't really see it as a fallback :laughing: It works fine, I think the sharding can work better and waste less resources spinning up a runner for each test instead of spinning up a consistent amount per PR.