I have recently moved from NXF V23 to v25 (server based) and am seeing a lot more files created and disk space used when running the pipeline. I can't seem to get a definitive answer as to whether this is really a result of the upgrade or something else in my pipeline. Anyone have any ideas?

Can you elaborate on what the files are?

Not sure really, the work directory for a run of 1 million id's contained 50,992 files (40G), but we had previously seen <15,000 for this number of id's. Seqera.A.I seemed to suggest there were more metadata files with v25 but I'm not sure how true that is.

Can you compare two task dirs and see what's different?

Unfortunately the work directory with the 15K files is long deleted (we have a tight inode limit). I haven't knowingly enabled Lineage. I just wondered, as we had done quite a lot of work to reduce intermediate file numbers and the inode count and disk space had been very manageable. Then, all of a sudden we saw a massive increase in inodes and, for one run, 230G of disk space use. We then switched to using

scratch = true

to combat this and it certainly decreased. All this seemed to correlate with a move from

v23

to

v25

but, of course, there may be other reasons.

Fair enough. I would probably try a small diagnostic run with both versions to see if you can replicate it and narrow it down.