Hello! I put this message in #CE6SDBX2A earlier and was kindly directed to this channel. If anyone could help me with this issue I have been having with phydamage not exiting I would greatly appreciate it!

Hey! I'm not an expert on the aDNA branch (@James Fellows Yates is), but I think it's probably because of the overrides. I did a quick search and it seems that pydamage memory usage scales with threads, and probably some forks are dying without reporting to the parent and that leads to that weird error. So, try removing the CPU override or lower it to something lower (default is 6). Regarding GTDB-Tk and CheckM, did them fail in your run? GTDB-Tk in theory shouldn't use more than the 140GB defined in the config, and definitively won't use more cpus effectively. In my tests the latest version uses very low CPU. If you want to speed up GTDB-Tk, check

gtdbtk_single_job

that runs a single job with all the bins, that could make better use of a larger number of CPUs.

Uhh I've never seen that before, and 18m reads is nothing... and to my knowledge, pyDamage should not need a huge amount of memory - I think something else is going on in there

I keep getting this error over and over again, I haven't been able to successfully run mag on this sample yet. There is no error message on the command log, this is the final line:

Estimating and testing Damage: 100%|██████████| 397687/397687 [2:42:00<00:00, 40.91it/s]

Hrmmm

Try keeping the memory high but lowering the CPUs (or leaving the default)

Yeah maybe try what Diego says...

Okay I will try that, thank you both!

Caused by:
  process hasn't exited

This is really odd tbh

The whole run? 19 hours

No just the pydamage process that fails

I believe so, I am running on my university HPC and submitting with the following resource requests: #SBATCH --time=960000 #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --cpus-per-task=32 #SBATCH --mem=256g #SBATCH --tmp=256g

Hrm, and are the nextflow jobs being submitted to the cluster as independent jobs (i.e., with

executor = slurm

) in a config somewhere?

Yes an independent job

Sorry I mean the steps of the pipeline

The second one. I submit the whole nextflow run as a single job. I made sure it was in a new environment where no previous nextflow job have run before

OK, then disregard my questions - go with Diego's. I was wondering if you were submitting Nextflow in the normal way where each step of the pipeline is submitted as independent jobs, then maybe there was some walltime setting somewhere that would kill it. But I think Diego's intuition is the best for now

Okay. I will let you both know how it goes