A quick question on reusing existing indexes in a pipeline.. Lets say I have a custom panel with a lot of genes spanning several chromosomes, and i want to subset these from the whole genome and maintain a local copy of the

panel_ref.fna.gz, known_snps.vcf.gz

, samtools faidx indexes, GATK4_COMPOSESTRTABLEFILE, TABIX_TABIX, BWA_INDEX. How do i set up these references for reuse in the pipeline. The aim is to have these in a blob where they can be applied as params into the pipeline. From running my pipeline a couple of times, re-indexing the reference takes a good 30+ minutes as e.g BWA_INDEX is not multithreaded. NB: a pointer to 1. How to integrate this into main nextflow.config 2. How to initialise the params in workflow, especially the bwa_indexes Here is my config..

cat assets/references.config
/*
========================================================================================
    SHARED REFERENCE GENOME CONFIGURATION
========================================================================================
*/

params {
    references_base = '/data/references/human_references'
}

// Internal databases
def databases = [
    clair3_model_hg38:   '${params.references_base}/hg38/clair3_model/clair3_model_dir/r1041_e82_400bps_sup_v420',
    pypgx_bundle:        '${params.references_base}/hg38/pypgx_bundle/pypgx-bundle',
]

// Internal presets
def reference_configs = [
    hg38_twist_panel: [
        description:  "hg38 Twist enrichment panel - targeted sequencing",
        genome_build: "hg38",
        use_case:     "twist_panel",

        fasta:        "${params.references_base}/hg38/twist_panel/fna/panel_targets.fna.gz",
        fasta_fai:    "${params.references_base}/hg38/twist_panel/fna/panel_targets.fna.gz.fai",
        fasta_gzi:    "${params.references_base}/hg38/twist_panel/fna/panel_targets.fna.gz.gzi",

        dict:         "${params.references_base}/hg38/twist_panel/gatk4/dictionary/panel_targets.dict",
        str_table:    "${params.references_base}/hg38/twist_panel/gatk4/str_table/panel_targets.fna.zip",

        dbsnp:        "${params.references_base}/hg38/twist_panel/dbsnp/panel_targets.dbsnp.vcf.gz",
        dbsnp_tbi:    "${params.references_base}/hg38/twist_panel/dbsnp/panel_targets.dbsnp.vcf.gz.tbi",

        bwa_index:    "${params.references_base}/hg38/twist_panel/bwa/panel_targets.fna",
        skip_bwa_indexing: true,

        target_bed:   "${params.references_base}/hg38/twist_panel/bed/panel_targets.bed",
        genome_sizes: "${params.references_base}/hg38/twist_panel/bed/genome_sizes.tsv",

        clair3_model: databases.clair3_model_hg38,
        pypgx_bundle: databases.pypgx_bundle,
    ]
]
// Auto-apply preset
if (params.reference_preset && reference_configs.containsKey(params.reference_preset)) {
    def config = reference_configs[params.reference_preset]
    config.each { key, value ->
        if (!params.containsKey(key) || params[key] == null) {
            params[key] = value
        }
    }
}

What I would recommend in such cases would be actually to use the igenomes.config file. It is actually meant to be modified, and you only really need just a basepath to point towards the files you're interested. The default one list many species, but not many files, and for me has always been meant to be modified and adapted.