Hi all. Just a quick question. I am running the nf-core/mag pipeline. Sometimes a job task will fail and I'll go to the working directory for that task to investigate. However, the input files won't be in that directory. I assume they're in another directory within the work/ directory, since the input files are created by a different upstream task. Is there an easy way to trace back and find them? Thanks! 🙂

Depending how at what scale you need to track this, when I'm in a particular work directory I just look at the output of

ls -l

which should tell you the source of the symlinked input file

Hey James, thanks for the reply. I thought a bit more and I think the problem is something to do with the process.scratch = true option in my config file. I enabled this to take advantage of the node storage (Im on an HPC) rather than writing to the disk. Thus, the symlinks are getting written to /scratch on the node rather than in the work dir first.

Ah got you

Wow, thanks for the thorough help! 'll look into your suggestions 🙂

For one-off stuff the pipeline table is probably easiest 🙂