Hey everyone! I'm struggling to get #CE9MS66BS v5.5.0 AWS full test running. Any AWS/Fusion magicians around? 🙏 Tasks fails with exit 175 or 174. In both cases the task doesn't seem to be running at all. From the

.fusion.log

of a task that exited 175:

22:18:06.266  configuration
22:18:06.346  FusionFS mounted at /fusion
22:18:06.85   .fusion/tmp/criu.kdat + .criu.move_mount_set_group.* probes
22:18:06.85   .fusion/dump/dump_metadata  (Insert entry)
22:18:06.953  FUSE server shutdown

The workdir has no

.exitcode

, no `.command.out`/`.command.err` and no partial outputs, so it's like

.command.sh

never executed. The module is a bioawk script that convert assembly depth format: github.com/nf-core/mag/blob/…/main.nf. Initially the module uncompressed the file inplace and then ran a for loop for every column. I changed that to pipe the script directly on

gzip -cd

and write the output file in one go to make it lighter / faster, but that did work either. Strangest part is that it's consistent for short-read assemblies while long-read ones finish fine, and both routes are identical, same module, same container, only depth file changes.

Does it only fail here and works on your local cluster?

Yes, CI works. And also a run in my HPC too (not exactly with the same data, but real data, no mock datasets).

can you check if it works on your end with the exact same data?

Ok, I'll do that

This run also had 9 spot reclamations that were saved so maybe fusion couldn't rescue the last one, but just want to make sure we know the data works

Ok, I'll confirm as soon as I have that process finished at least

I had resubmitted it on a hunch on the on-demand CE to avoid more spot reclamations and this looks promising. It stil has a few failures, since they have errostrategy ignore, is this expected?

Hi again! The resubmit failed on CONVERT_DEPTHS too, same as before. The run on my HPC still doesn't finish but is running the last binning jobs (some of them retried by time). Regarding CONVERT_DEPTHS, all jobs finished successfully on first try with basically zero footprint (<=2s runtime, <30MB RAM usage). Very strange...

thanks for confirming. I'll take another look today

It finished now, here are the report and trace in case it is useful. Let me know if you need something else.

Did @James Fellows Yates try this one: https://nfcore.slack.com/archives/CE9MS66BS/p1786439539376239?thread_ts=1786437612.532309&amp;cid=CE9MS66BS

Not yet, he isn't available right now

@Rike Hanssen I have a spare 2 minutes, do you mean duplicate the exisitng non-fusion one I tried, but add a chunky number here:

yep

Ok, any guide of what I should set?