I launched the #C08THK11CHX tests and they appear as submitted but they never start running any ideas or what could be going on?

Can someone confirm that the AWS Batch compute environment used for full-size tests is configured with c6id, m6id, and r6id instance types with Intel Xeon Platinum 8375C (Ice Lake) CPUs? This is the answer I got form the Sequera co-scientist but I would like to double-check, because I want to use this information in a paper where I will use the #C06TRU74N4D pipeline's full test as a benchmark. By the way, the AWS full tests are really cool…

The AWSmegatests should be back up and running. If you have released a pipeline in the last couple of days without running the full size test, please retrigger the action for the released tag. Let me know if you run into any issues.

I am putting in a hackahton project to overhaul these: github.com/nf-core/website/pull/4376 let me know what else you'd like to see. It's already way to ambitious for 2 days, but hopefully we can keep some momentum after the hackathon to close out the big ones

In the pairgenomealign 3.4.0 pipeline, the AWS test were triggered when my release PR to the main branch was accepted, and they were successful, as announced in the #C06TRU74N4D channel. Then I released the pipeline, and I got an email that they failed, but the logs (github.com/nf-core/pairgenomealign/…/33025134131) suggest that the problem is not on the pipeline's side. The nf-co.re/pairgenomealign/3.0.4 website says that the tests were never run. What is the usual way to trigger them again?

Hi everyone! I merged the release PR for genomeqc 1.0.0. The AWS megatest was triggered, but it’s failing at the OrthoFinder step. It looks like the issue is with the

mv

command in the orthofinder module. In the AWS megatest,

mv

isn’t just renaming the output folder, instead it's doing a per-object copy-then-delete. The problem is that the output folder contains several subfolders and files, so it fails. Is there a way to bypass this? Or would the only option be to patch the module in the pipeline and remove the

mv

line. I tried re-running the megatest and get the same error.

Is data for the megatests typically staged into some s3 bucket? (My external download failed)

My megatest errored, and I dont think it is my pipelines fault cloud.seqera.io/orgs/…/logs what can I do about this? As far as I can tell, the file that cannot be found exists cloud.seqera.io/orgs/…/tasks?taskId=71

Megatest failed again, but with a different problem:

1:03PM INF [seqera-fusion] sending incremental snapshot signal (SIGUSR1)
[2026-09-10 13:03:14.845] [WARN] Warn  (dumprestore.c:235): predump failed, starting new chain for future dumps
[2026-09-10 13:03:14.846] [ERROR] Error (exec.c:102): incremental dump failed with code: -52
make: *** [/usr/local/bin/LINKS-make-real:122: test_hifi_hifiasm_links.tigpair_checkpoint.tsv] Killed
FUSION_GPU_USED=false
Fusion Info:
    available_disk: start:2.5TB, min:2.4TB, max:3.5TB, end:3.5TB
    clone_namespace: false
    download: 47210.36MB, drops:0, skip:0, sync:0, req:370
    fusion_version: 2.5.14-20e904e
    kernel_version: 6.1
    max_open_files: 1048576
    upload: 169466.41MB, drops:0, skip:0, sync:79, req:1402

I do not really know what this means tbh

Hi all, I had requested a database upload to the s3 bucket, but forgot that a command needs to be run to format and index the database before it can be run by the tool using it. should I: a) somehow get shell access to an aws compute environment where I could log in and run the commands to format the database? b) request a re-upload from someone over at request-core, asking them to do the formatting command for me? c) adjust the pipeline to conditionally format the raw database if thats all thats provided? I'd rather not do c, if it can be avoided 🙂

To start an aws test run, if my previous run failed, will starting this run with

relaunch

pull from the current tip of dev, or will it pull the commit-sha that was used on the failed run? My impression is that it is the latter. If that is correct, how can I start a run from the current tip?

Hi, I noticed that the aws workflow results that are published on nf-co.re are missing the report. The process ran during the test and produced a file, but it does not show in the AWS results. Is there some configuration that I missed?

For the next release of ampliseq I wanted to test whether AWS runs are still fine. I submitted

nf-core AWS test

on dev (not "full size tests" to minimize test costs), but it failed: github.com/nf-core/ampliseq/…/110261116126. I have no idea of Platform/AWS, so I am a bit lost. Shall I just re-try? Shall I run

nf-core AWS full size tests

on dev instead? Edit: as far as I see no run was triggered on platform

Fusion spot snapshots break processes that read through a pipe (AWS Batch, nf-core megatests) The nf-core/metatdenovo AWS full test failed three times in a row, each time within seconds of Fusion's incremental snapshot signal (

sending incremental snapshot signal (SIGUSR1)

, every 5 minutes, spot compute environment). Each failing step was reading data through a pipe at that moment: - BBMap (twice): BBTools decompresses BGZF input via a

bgzip -c -d -@ 4

subprocess. bgzip logged

Could not write 65280 bytes

(one BGZF block lost), and BBMap then failed with

Mismatch between length of bases and qualities

. The same files map without errors locally with the same container and arguments. - Prokka (once): the HAMAP step,

cat ... | parallel --pipe hmmscan

, died right after the snapshot signal. Setting

fusion.snapshots = false

in the launch config fixed both: in the next full test, all BBMap and Prokka tasks passed, without any change to the tools (nf-core/metatdenovo#584). Environment: Nextflow 26.04.6, nf-amazon 3.9.2, Fusion v2.5. Whatever the snapshot does to the process tree seems to interrupt pipe writes between processes. Happy to provide task IDs, timestamps and work directories. (Analysis done with Claude Code.)

Trying to remedy the above and get a working full test for yesterday's 2.0.0 release of metatdenovo I'd like to relaunch the run "elated_mccarthy" from yesterday but I can't see that I can. This is what Claude tells me regarding that: It needs two changes, neither of which alters task hashes: - Nextflow config:

fusion { snapshots = false }

(Fusion snapshots broke piped steps; fix for future runs in nf-core/metatdenovo#584) - Parameter:

"skip_eukulele": true

(EUKulele tries to write a Diamond database into the read-only

ngi-igenomes

bucket, nf-core/metatdenovo#585) Same revision, outdir and work directory, with resume on, so the results land in

results-e9f8311…

and show up on the 2.0.0 page.

I'm getting this error on the full test of rnasplice dev:

Missing process or function paramsSummaryLog([[:], repository: <https://github.com/nf-core/rnasplice>, projectDir: /.nextflow/assets/.repos/nf-core/rnasplice/clones/bfa5aa043c60a1ec1d515306f958e39b0fc11fd2, commitId: bfa5aa043c60a1ec1d515306f958e39b0fc11fd2, revision: null, startTime: 2026-10-10T11:52:50.716082839Z, endTime: null, duration: null, container: {}, commandLine: nextflow run '<https://github.com/nf-core/rnasplice>' -name crazy_ardinghelli -params-file '<https://api.cloud.seqera.io/ephemeral/kSF8hg9m702f194s_d5ASg.json>' -with-tower -r bfa5aa043c60a1ec1d515306f958e39b0fc11fd2 -profile test_full, nextflow: nextflow.NextflowMeta(version:26.04.0, build:12031, timestamp:29-04-2026 04:00 UTC, preview:nextflow.NextflowMeta$Preview@7bbec1cc, enable:nextflow.NextflowMeta$Features@5a77b7cc, strictModeEnabled:false, moduleBinariesEnabled:false), success: false, workDir: /nf-core-awsmegatests/work/rnasplice/work-bfa5aa043c60a1ec1d515306f958e39b0fc11fd2, launchDir: /, profile: test_full]) -- Did you mean 'paramsSummaryLog' instead?

I'm using nf-schema@2.7.2 but it seems that AWS workflow passes its plugin

plugins {
   id 'nf-slack@0.5.0'
}

and that overrides it, so I have to manually add nf-schema to awsfulltest.yml Is that something you have seen before? Is my edit correct or there's another way to fix that? Thanks!

@Edmund @mashehu at the hackathon this week we must purge the old action-tower-launch and replace it with a typescript version

syntax parser should be available via the API: https://cloud.seqera.io/openapi/seqera-api-latest.yml

hello, not in barcelona, but saw the megatests overhaul project. i've been going through the rnaseq full-size runs in the public bucket, and a few things stood out: - rnaseq 3.20.0 and 3.21.0 have no full-size results under their release commits, and 3.27.0 only has star_rsem. none of the three are on the missing megatests canvas yet - star_rsem failed in 3.22.1 and 3.23.0 to 3.25.0 because RSEM hit its 16h limit, and in 3.27.0 one RSEM task only finished on its third attempt, at 36 CPU / 216 GB. median use is about 1 core (rnaseq#1957, fix open in rnaseq#1959) - on star_salmon, 62-68% of the requested compute went unused in 28 of 30 releases write-up: github.com/OtoYuki/nf-audit/blob/…/rnaseq-61-runs.md i'd be glad to run the same over other pipelines' full tests if that helps anyone : )

@Rike Hanssen Testing the new action in sarek github.com/nf-core/sarek/pull/2320!

@Rike Hanssen github.com/nf-core/ops/pull/235 PR for you to work off

<!here> I have updated everyones permission and moved you all into teams. If anyone lost access, or can't do things they could before, please reach out. In principle: • Everyone is now a member of a team. The reason we decided on this is, that it makes permission & user management much much easier. • Teams: ◦ core: ▪︎ who? core team ▪︎ permissions? god -powers ◦ pipeline-maintainers: ▪︎ who? everyone that had platform access before and is actively maintaining a pipeline ▪︎ permissions? new shiny maintainers custom role, that is the old launch + adding pipelines to the launchpad and being able to modify the nextflow config in the pipeline. This should a) allow you to add pipelines to the launchpad (we have more automation planned for this) & b) you can set things like scratch/fusion version/ queue names etc yourself now. ◦ viewers: everyone else, just view permissions <!subteam^S016E36N7TQ> if you add new people manually please use the above groups here. I am working on a system that uses slack & github groups to automate the grouping further, so keep an eye out for that.

It's live! registry.nextflow.io/plugins/…

wrap-up slide: https://docs.google.com/presentation/d/1tT_pGwbTZFUvSlqL8jSH6g3izeshUT7Z93E2H2GSSUI/edit?slide=id.h2ccdff66c22b2d4c_44_0#slide=id.h2ccdff66c22b2d4c_44_0

Lineage is now enabled for all runs. this should help with two things here in particular. If results are unexpected you can trace back where they came from more easly, same when the pipeline fails. For pipelines that use output syntax the files are also listed in platform and are more easily accessible.

I relaunched a previously failed run for ampliseq 3.0.0 with parser v2, but it failed again cloud.seqera.io/orgs/…/logs with

Caused by:
  Wave invalid response: POST <https://wave.seqera.io/v1alpha2/container> [400] {"message":"Container image '<http://quay.io/biocontainers/hmmer:3.3.2--h87f3376_2|quay.io/biocontainers/hmmer:3.3.2--h87f3376_2>' does not exist or access is not authorized"}

is this something that needs to be fixed on the pipeline side?

Seeing some errors in megatests

Wave invalid response: POST https://wave.seqera.io/v1alpha2/container [400] {"message":"Container image 'quay.io/biocontainers/***' does not exist or access is not authorized"}

I have a feeling this is either quay.io or random infra failure

If your pipeline is using the v2 parser, the megatest submission as it is at the moment, will fail. We have updated the action and added v2 parser support plus a number of other fixes and QOL elements. Here is an example of how to use it: github.com/nf-core/airrflow/pull/519

@Rike Hanssen have you updated the docs on what the new system is for adding new pipeline maintainers to the megatest workspace (either during piepline acceptance, or alter?)