Does anyone run parabricks on batch and if yes, wh...
# parabricks
e
Does anyone run parabricks on batch and if yes, what marketplace AMI would you reccomend for running parabricks (on GPU hardware) in AWS batch?
a
So: • NVIDIA docs on AWS suggest that a “DeepLearning” AMI be used, but this is not talking about Batch necessarily • This penultimate section of this Seqera blog suggests that they use the Amazon recommended GPU-optimized AMI When I look at the NVIDIA AMI on the catalog it seems it has a lot of extra cruft pre-installed (miniconda, jupyter, etc). Unless the Seqera Platform option for using the latest Amazon recommended GPU is not actually using that AMI and is secretly doing some other magic behind the scenes (Platform is closed source so not sure what the option really does) then I would suggest that you can just use the default Amazon recommended image, which as far as I know is actually the default when using a managed Batch compute env. (Hence why I am somewhat suspicious of what the Platform option does). Relevant AWS docs:
In managed compute environments, if the compute environment specifies any
p3
,
p4
,
p5
,
p6
,
g3
,
g3s
,
g4
,
g5
, or
g6
instance types or instance families, then AWS Batch uses an Amazon ECS GPU optimized AMI.
TL; DR: Try running with the default Amazon GPU AMI. (Usual nextflow caveats about needing to inject the AWS CLI into your images applies, but if you are already running non-GPU workflows on Batch then I assume you have a chosen solution for that?)
e
Thanks, will look into these.