A community effort to collect a curated set of analysis pipelines built using Nextflow.

nf-core

And, on a related note, I am having issues with caching for `-resume` . I restarted the run without touching anything, and it only loaded the preprocessing steps from cache, and started processing from BWAMEM1_MEM. What is going on there?

:screams: Idk but no good. can you share some logs

If you don't have logs, is this reproducible  on for example the full size test with the same parameter configuration?

I have the logs, I will try to get them to you, it might just take a while

Any news on this? The issue keeps appearing in other runs with different data, and it always resumes specifically from BWAMEM1_MEM. I don't know if it's the fault of my setup or something else.

Can you test with `cache: lenient` ? Claude seems to think that this is caused by the file system. In addition, you could also add `-dumpHashes` which would ive us more info on which hashes are mismatching

```That is the diagnostic signature of input-staging hashes drifting between runs. Nextflow's default cache mode hashes each input by path + size + last-modified-time. The
  samplesheet/reference files don't change those, so they hit. The work-dir-staged outputs do — meaning size or, much more likely, mtime of the staged files is changing between sessions.

  The work dir is &lt;redacted&gt;/work [fuseblk] — CERN-style EOS over FUSE. EOS-via-FUSE is well known for not preserving stable mtimes
  (replication, scrubbing, FUSE attribute caches, periodic GC can all shift them), and the NoSuchFileException on a collect-file entry confirms the FS is also losing/aging out files. Nothing
   in the BWAMEM1 module or the FASTQ_ALIGN subworkflow uses non-deterministic ext.args/ext.prefix that would explain a universal miss for all 12 lanes (the dynamic ext.prefix and ext.args
  resolve from meta.id, reads.get(0).name, meta.read_group, meta.status — all stable).````

It would like to validate this:
```Caveat: I'm reading this from logs only. To fully confirm I'd want to see two stat snapshots of e.g. work/a9/19ab54*/bwa/*.amb and work/31/af7180*/*.fastp.fastq.gz taken hours apart, plus
  the user's nextflow.config from the launch dir to rule out an unusual process.cache override. If those are stable, my hypothesis is wrong.```
if you are able to check the files

I have trying to reproduce this and could not, so I am tempted to side with claude on blaming the infrastructure. Are you able to reproduce this issue on a different system, like your laptop for example?

also this is the rec for the infra in general, but I can't speak to this as I don't know your setup well enough:

```  3. Move workDir off EOS-FUSE to local/scratch storage — best long-term fix. Keep only outdir on EOS via publishDir.```


Alright, I will test the ideas when I have a moment

Good and bad news, I checked on my last run and it succeeded this time, so I might take more time to investigate