# pipeline-resource-optimization
- d
Damon-Lee Pointon
03/24/2025, 11:50 AMHi all, The question about dynamic optimisation has come up at sanger again, my mentee's made good headway with the whole thing. We did however run out of time. Would there be interest in a interest_group meeting to talk about everyones approaches so far and how we can go ahead in the future? If so, when would be best for people? Should I start a poll? Thanks all.high five 1sm+2- 5
- 7
- s
Slackbot
06/22/2025, 12:21 AMWellcome Sanger Institute has been disconnected because all members from that organisation have left this channel. - f
Florian Wuennemann
10/28/2025, 10:09 AMThis will be the channel the resource optimization group at the hackathon in Barcelona communicates in!🙌 2mr- 3
- 2
- n
Noah Jornet Calomarde
10/28/2025, 10:10 AM - r
Rashmi Naidu
10/28/2025, 10:13 AM🎉 - f
Florian Wuennemann
10/28/2025, 10:17 AMSlide deck from this morning with links to existing repos: https://docs.google.com/presentation/d/1chSH6DNEjwArolFI57rMZi5_T73alDz48q8xR2ucF3w/edit?usp=sharing - r
Regan Hamel
10/28/2025, 10:17 AM - r
Rashmi Naidu
10/28/2025, 10:20 AM - r
Rashmi Naidu
10/28/2025, 10:29 AMhttps://github.com/seqeralabs/tower-cli - Tower CLI https://docs.seqera.io/platform-cloud/seqerakit/installation - SeqeraKit CLI (wrapper around tower CLI) https://docs.seqera.io/platform-cloud/seqerakit/commands - general Seqera Docs - f
Florian Wuennemann
10/28/2025, 10:53 AMFor @Mauro Saporita for CO2 calculations, one potential tool to use: https://github.com/GlasgowC3lab/ichnos - f
Florian Wuennemann
10/28/2025, 11:40 AM@Regan Hamel @Noah Jornet Calomarde CO2 plugin: https://nextflow-io.github.io/nf-co2footprint/ - f
Florian Wuennemann
10/28/2025, 11:54 AMThis is the API endpoint to get an optimized configuration :Copy codecurl '<https://cloud.seqera.io/api/workflow/4Gxsym6QSrqSvS/optimal?workspaceId=80750985193419&optimizationTargets=cpus,memory>' \ -H "Authorization: Bearer $TOWER_ACCESS_TOKEN" | jq -r '.config' - n
Noah Jornet Calomarde
10/28/2025, 1:34 PMpython get_megatest_workflow_configurations.py -w 80750985193419 - r
Rashmi Naidu
10/28/2025, 2:25 PM - f
Florian Wuennemann
10/28/2025, 2:32 PM - r
Rashmi Naidu
10/28/2025, 2:32 PMtw launch -w 80750985193419 --params-file /Users/rnaidu/Desktop/hackathon_2025/seqera_cloud_runs/test_full.config -plugins nf-co2footprint@1.0.0 - m
mashehu
10/28/2025, 3:18 PM - r
Rashmi Naidu
10/28/2025, 3:24 PMprocess { resourceLimits = [ cpus: 2, memory: '4.GB', time: '1.h' ] } params { config_profile_name = 'Test profile' config_profile_description = 'Minimal test dataset to check pipeline function' // Input data input = 'https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv' } - f
Florian Wuennemann
10/28/2025, 3:25 PM - r
Rashmi Naidu
10/28/2025, 3:47 PM${workflow.runName} - f
Florian Wuennemann
10/29/2025, 9:46 AM@Noah Jornet Calomarde This is the seqerakit block you can use to define an outdir:Copy code
let's add pipeline_name/test or test_full and optimized / not_optimized afterparams: outdir: "<s3://nf-core-awsmegatests/resource_optimizations/>" - f
Florian Wuennemann
10/29/2025, 9:57 AMCO2 plugin activation @Noah Jornet CalomardeCopy codeplugins { id 'nf-co2footprint@1.0.0' } // Optional example config settings for COâ‚‚ reporting: def co2_timestamp = new java.util.Date().format('yyyy-MM-dd_HH-mm-ss') co2footprint { traceFile = "${params.outdir}/pipeline_info/co2footprint_trace_${co2_timestamp}.txt" summaryFile = "${params.outdir}/pipeline_info/co2footprint_summary_${co2_timestamp}.txt" reportFile = "${params.outdir}/pipeline_info/co2footprint_report_${co2_timestamp}.html" location = 'IE' } - f
Florian Wuennemann
10/29/2025, 10:03 AMChanging to writing everything to:<s3://nf-core-resource-optimization>m- 2
- 1
- f
Florian Wuennemann
10/29/2025, 10:04 AMwork-dir: "s3://nf-core-resource-optimization" - n
Noah Jornet Calomarde
10/29/2025, 11:09 AMwork path for rna pipeline fail /nf-core-awsmegatests/resource_optimizations/work - n
Noah Jornet Calomarde
10/29/2025, 11:14 AMcli.yml launch: - name: "rnaseq-test" workspace: "nf-core/ResourceOptimization" compute-env: "aws_ireland_fusionv2_nvme_cpu_snapshots" pipeline: "https://github.com/nf-core/rnaseq" revision: "3.21.0" labels: "testPreOpt" profile: "test" work-dir: "s3://nf-core-awsmegatests/resource_optimizations/work" params: outdir: "s3://nf-core-awsmegatests/resource_optimizations/rnaseq/test/not_optimized" - f
Florian Wuennemann
10/29/2025, 1:44 PMSo it begins, end sprint: - r
Rashmi Naidu
10/29/2025, 4:20 PMHi all! Here’s my LinkedIn if you want to connect: LinkedIn.com/rashmi827 - r
Rashmi Naidu
11/04/2025, 4:04 PM - j
Júlia Mir
02/13/2026, 9:23 AMHi! I was fixing my pipeline (#C03PADVJ8RL) full tests and I think it's a good moment to try to optimize resources, do you have some pointers on how to find the resources I should use? I seem to remember there was a way to obtain this form the Seqera platform but I can't find ite- 2
- 5