Fusion spot snapshots break processes that read through a pipe (AWS Batch, nf-core megatests) The nf-core/metatdenovo AWS full test failed three times in a row, each time within seconds of Fusion's incremental snapshot signal (

sending incremental snapshot signal (SIGUSR1)

, every 5 minutes, spot compute environment). Each failing step was reading data through a pipe at that moment: - BBMap (twice): BBTools decompresses BGZF input via a

bgzip -c -d -@ 4

subprocess. bgzip logged

Could not write 65280 bytes

(one BGZF block lost), and BBMap then failed with

Mismatch between length of bases and qualities

. The same files map without errors locally with the same container and arguments. - Prokka (once): the HAMAP step,

cat ... | parallel --pipe hmmscan

, died right after the snapshot signal. Setting

fusion.snapshots = false

in the launch config fixed both: in the next full test, all BBMap and Prokka tasks passed, without any change to the tools (nf-core/metatdenovo#584). Environment: Nextflow 26.04.6, nf-amazon 3.9.2, Fusion v2.5. Whatever the snapshot does to the process tree seems to interrupt pipe writes between processes. Happy to provide task IDs, timestamps and work directories. (Analysis done with Claude Code.)

Thanks for sharing this, I think that I am seeing something similar with some modules in the AWS tests.

@Daniel Lundin the team is checking. Could you please link the run

Is the run name "elated_mccarthy" sufficient information? I think I might have missed sending one message in which I asked you to resume this with

--skip_eukulele

added as param besides the

fusion { snapshots = false }

.