Hi all. I want to better understand how nf-core subworkflows are tested on the GitHub CI. I notice a lot of breaking changes keep getting merged and go undiscovered until you go to update one of the modules they touch. To me, it looks like the CI runners aren't running the subworkflow tests properly, but I want to know if this is expected behavior. For context, see this issue. @Kevin Brockers for viz

Hmm, they should be running. In this case, it looks like the gpu test was initialised, but no tests were executed: https://github.com/nf-core/modules/actions/runs/23059334528/job/66983111188#step:5:972 So perhaps there is an issue on the CI runner setup there. @mashehu?

Thanks for the extra context. I was surprised to see that I fixed one subworkflow, merged it into my module PR, and promptly discovered a different subworkflow is failing, too. I'm wondering why that second subworkflow didn't fail during my initial fix (AKA the tests didn't run). It resulted in me submitting another PR when I would have expected to catch both of them in one go.

Worth flagging I shipped two fixes to GPU CI a couple of weeks ago that are directly relevant: • https://github.com/nf-core/modules/pull/11179 (Apr 14) aligned the sharding step's tags with the detection step. Before, detection used

gpu,gpu_highmem

but sharding only used

gpu

, so tests tagged solely

gpu_highmem

(parabricks) got detected, produced 0 shards, and the runner ran nothing. • https://github.com/nf-core/modules/pull/11203 (Apr 16) split GPU CI into independent

gpu

and

gpu_highmem

jobs so that class of bug can't recur. Checking the runs: • Simon's example (https://github.com/nf-core/modules/actions/runs/23059334528, Mar 13) is on a commit that pre-dates both fixes. That "initialised, no tests executed" log is exactly the silent-skip bug they fix. • Gary's PR https://github.com/nf-core/modules/pull/11217 was opened Apr 17, after both fixes landed. The GPU CI run there (https://github.com/nf-core/modules/actions/runs/24574690745) actually failed and caught the bwamem input mismatch from https://github.com/nf-core/modules/issues/11238. Working as intended. • The bwameth case (https://github.com/nf-core/modules/pull/11434) is a different gap. Gary's earlier fix PR https://github.com/nf-core/modules/pull/11362 only touched

fastq_align_dedup_bwamem/

and

fastq_align_bwa/

files, no bwameth files in the diff. So change-detection had no reason to queue bwameth tests, and they weren't run. The runner didn't silently skip them; they were never picked up.

Thank you so much, Jonathan for bringing up your recent GPU CI changes. It looks like Simon's example, which was merged before your fix, explains the behavior I was seeing and I don't think it should happen again.