Hi <!here>, We are facing an issue with GPU availability when running workflows via Seqera platform using awsbatch, even though the same setup works correctly when executed manually on a GPU instance. This issue is specifically observed in #CHN5BV5DW CellBender background removal step, which uses the process_gpu configuration. What We Tried 1. Instance Identification ◦ compute using the ▪︎ AMI: Amazon Linux AMI 2023.0.20260424 x86_64 ECS HVM EBS ▪︎ Instance type: g4 (GPU-enabled instance) 2. Manual GPU Instance Testing ◦ Using the same AMI, we launched a GPU instance manually. ◦ Verified execution locally: ▪︎ Ran the container interactively using Docker → ❌ Failed (CUDA not detected) ▪︎ Observed the same CUDA issue with the official nf-core GPU image. 3. Nextflow Local Execution ◦ Attempted to run via Nextflow → ❌ Same issue (

CUDA is not available

). 4. Fix Applied ◦ Identified that GPU access required explicit configuration. ◦ Added: --gpus all via

containerOptions

in Nextflow config. ◦ ✅ This resolved the issue: • Docker (interactive) → Working • Nextflow (local) → Working 5. Seqera Platform Execution ◦ Applied the same configuration in Seqera. ◦ ❌ Issue persists: GPU is still not detected (

CUDA not available

). 6. Additional Checks ◦ Verified accelerator operator → Already present ◦ Tried multiple configurations like containeroptions, docker.runoptions for that particular process → No success ❗ Current Issue Even after: • Using the same AMI, • Ensuring GPU instance type, • Passing

--gpus all

, • Confirming local success with Docker and Nextflow, GPU is not accessible when the pipeline is triggered from Seqera. Question: What could be causing GPU not being exposed/accessible inside containers when launched via Seqera?

Maybe @Florian Wuennemann or @Rashmi Naidu can help here? I know they (and a few others) have been running a lot of GPU workloads via Platform recently..

@SINRASU ALAGUDURAI sorry to hear you are having issues getting gpus to work on Seqera Platform. From reading through your thread, the very first question I have is, did you enable the GPU AMI during CE creation? A quick check of your AMI looks like its a cpu only AMI. So my best guess is that CUDA is not available because the AMI isn't GPU supported.

@Florian Wuennemann, I have also verified the CE, ensuring that GPU support is enabled. Additionally, we confirmed that the AMI being used supports GPU workloads by launching a similar GPU instance manually and running the Nextflow pipeline successfully. The AMI ID:

ami-03514758ac1381b2f

AMI name:

al2023-ami-ecs-gpu-hvm-2023.0.20260424-kernel-6.1-x86_64-ebs

Ok, so first theory down then 😅

Hi @SINRASU ALAGUDURAI, would you feel comfortable sharing the log file from the most recent failed run executed on Seqera Platform, or a screenshot of the error message you're getting from your console output? Since you've applied the same configuration, I'm curious to see if Platform is spitting out a clear error which may help us get to the root cause of what's happening!

Hi @Rashmi Naidu, I am sharing the console log file here.

@SINRASU ALAGUDURAI Did you pin the GPU AMI option but still select manually a GPU enabled AMI or let Platform auto-select a GPU AMI for you?

@Florian Wuennemann, We haven’t explicitly specified any AMI. I’ve attached the relevant part of the CE configuration for reference. Also, we have tested with different instance types such as g4, g5, and g6.

@SINRASU ALAGUDURAI Can you re-launch using the gpu profile added to the "Config Profiles" field? You can find this in the launch form under "1) General Config." This should apply the

accelerator=1

directive and trigger the AWS Batch GPU resource request. Let me know if this works

@Rashmi Naidu We have configured a GPU-specific profile for this process and executed the pipeline. From the logs, i can confirm that a GPU-enabled instance(g4 type) was used for this step. I am attaching the relevant screenshots here for your reference along with Metrics configuration used for the failed process CellBender background removal.