I've had an issue with getting ribodetector to work in Apptainer/Singularity (using latest pipeline release 3.27.0 or prior).
I'm aware of a previous issue with ONNX multiprocessing hang, that had to do with previous ribodetector versions but supposedly this has been fixed for ribodetector 0.3.3.
Even when adjusting '--chunk_size' to smaller values, running the pipeline with gpu ribodetector always tries to allocate around ~ 75GB to my GPU when I only have ~50GB available.
here is part of my config:
// withName: '.*RIBODETECTOR' {
// queue = 'gpu_h200'
// clusterOptions = '--gres=gpu:1'
// containerOptions = { workflow.containerEngine == 'singularity' || workflow.containerEngine == 'apptainer' ? '--nv' :
// ( workflow.containerEngine == 'docker' ? '--gpus all' : null ) }
// memory = { 120.GB * task.attempt }
// cpus = 8
// time = 24.h
//
// // This prevents the H200 from hitting a CUDA OOM
// ext.args = '-l 99 -m 180 --chunk_size 4096'
// }
For the cpu version of ribodetector, it just goes on forever and never finishes (which makes no sense).
Should I try to allocate more GPU? sortmerna works fine in every run, which I have defaulted to because ribodetector has never worked for me.