https://nf-co.re logo
Join Slack
Powered by
ARM64 support - Looking to help a researcher run t...
# arm64
m
ARM64 support - Looking to help a researcher run the rnaseq pipeline on a Nvidia DGX Spark (128GB RAM) as 👍 Was able to run the test pipeline on DGX Spark consuming more than 16GB (M3 Pro) on my laptop
Copy code
nextflow run nf-core/rnaseq -r 3.22.2 -profile docker,arm64,test --outdir training/results_test
Note: I decided not to try on Mac Studio M4 Max (128 GB RAM), because my experience with getting Metal to use GPU on my M3 Pro laptop was pitiful. (related to my AI/ML aspirations)
👀 1
Now I got off to a rough start working from AI assisted research (reads the docs bro!), and had started from an older lescai-teaching/rnaseq-tutorial that I was able to run on gitpod.io and all. Somewhere along the way I ended up with https://github.com/msusol/rnaseq-docker/tree/main/research/data/reads and was able to process the pipeline on gitpod.io and then download the results and perform de_analysis on the results. But where did I get that dataset (?) and why doesn't it work now? (and why can't I edit my posts, only delete?)
Is there a current dataset that works w 3.22.2 to due a similar control/treatment anlaysis, just so I can ensure the R code is working. Will expose my DGX Spark to the researcher once we're ready for him to use
a
@Gary Burnett any pointers on running on DGX Spark?
m
i have gotten the pipeline to run on real data tonight (190GB). but needed to really tone down the 
base.config
. i don't know what machine its meant to run on, with 200GB RAM, 32 cpus. vs 3.23.0 docker,arm64 downloaded the GRCh38 genomes from s3 (30GB but s3 was super slow, it ran overnight)
Copy code
process {
    // NVDIA DGX SPARK: 20 cpu, 128 GB ram
    cpus   = 20
    memory = '110 GB'

    withLabel: 'process_high' {
        cpus   = 16 
        memory = '110 GB'
    }

    // This overrides any process tagged with the 'process_high_memory' label (200GB)
    withLabel: 'process_high_memory' {
        memory = '110 GB'
    }

    withName: 'FASTQC' {
        cpus   = 8
        memory = '32 GB'
    }

    // Ensure the human genome index fits
    withName: 'SALMON_INDEX' {
        cpus   = 16
        memory = '110 GB'
    }
}
one warning not sure about:
Copy code
WARN: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Biotype attribute 'gene_biotype' not found in the last column of the GTF file!

  Biotype QC will be skipped to circumvent the issue below:
  <https://github.com/nf-core/rnaseq/issues/460>

  Amend '--featurecounts_group_type' to change this behaviour.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 Views
Open in Slack
PreviousNext
Powered by