Hello, my plan is to run the nf-core/ampliseq pipeline on PDC from KTH for the first time. When I was using the pipeline on Uppmax last year, I’d just open a screen and load Nextflow with the following commands: module load bioinfo-tools module load Nextflow Then I’d launch the pipeline with a command like this: nextflow run nf-core/ampliseq -r 2.11.0 -profile uppmax -params-file nf- params.json --project naiss2023-22-1217 Since we are on Dardel now, I believe a few things have changed and thus, I have a few questions: Do I still work in a screen? How do I load modules like Nextflow on Dardel? The profile to launch the pipeline is now pdc_kth I guess? On Uppmax - for ampliseq - I never used a config file to set how much memory, cpus and time I want to allocate for certain tasks within the pipeline. But by reading through the comments of previous questions, I saw that economizing cpu cores seems to be much important on Dardel than Rackham. So, is this something I need to do (we have NAISS small compute project)? If yes, how do I do it? For example, how do I call a nextflow.config when launching the pipeline? Thank you for our help!

On dardel you need to do:

module load PDC apptainer

and run with the

-profile pdc_kth

. I usually load Nextflow with

pixi

or before that

conda

. It's fine to also just install the

nextflow

binary in your bin. Also normally run in a screen session too. On Dardel you don't need to remember which login node you're in though

Thank you for your answer! What would be the command to load Nextflow with pixi (or conda)?

Download Pixi from here: https://pixi.sh/latest/

curl -fsSL <https://pixi.sh/install.sh> | sh

Depending on the operating systems you're working on and if you're using version control to share an environment between then you would do: If you're only running nextflow on dardel, the leave off the

-p osx-arm64

( which would be for arm macs ).

# In your analysis project directory
pixi init -c conda-forge -c bioconda -p linux-64 -p osx-arm64
pixi add nextflow
pixi shell
nextflow run ...

pixi will write a

pixi.toml

file and a

pixi.lock

file which you should check into version control if you're using that. Then each time you want to run the pipeline, you can go to that directory, run

pixi shell

,

nextflow run ...

, and then

exit

. I'll stop there. Just ask more if you're curious on utilizing Pixi further.

Thank you!

Nextflow is also trivial to install without using conda or similar (single file fetch).

The reason I use Pixi for it is because sometimes I do encounter issues with particular versions of Nextflows, and it's handy to have it project based. I can explore new features in my sandbox, but if something messes up for a larger pipeline ( which I just spend the last month doing) being able to roll-back to an earlier version in the project helps - I could also set

NXF_VER

and that's tracked in the log so one can choose their medicine. Using Pixi though means the others on my team who might take over end up using the same environment.

Good morning, I've been using Mahesh's code using pixi (

# In your analysis project directory
pixi init -c conda-forge -c bioconda -p linux-64 -p osx-arm64
pixi add nextflow
pixi shell
nextflow run ...

) to run a next flow pipeline but then I encountered the following issue

bash: singularity: command not found

. What do I need to add to this, so it works?

module load apptainer

(or singularity, either should be fine).

Should be

module load PDC apptainer

if the

PDC

module is not loaded too

Okay. I did the following now. I used pixi shell and then I opened a screen session. In the screen session I used

module load apptainer

. But then I get the following error: Lmod has detected the following error: These module(s) or extension(s) exist but cannot be loaded as requested: "apptainer"

Add the

PDC

before

apptainer

That worked now, thank you! However, I now tried to run the ampliseq pipeline using the following command: nextflow run nf-core/ampliseq -r 2.12.0 -name Luuk2 -profile pdc_kth -params-file nf-params_Luuk.json --project naiss2023-22-1217. Now I got the following error: ERROR ~ Error executing process > 'NFCORE_AMPLISEQAMPLISEQFORMAT_TAXONOMY' Caused by: Failed to submit process to grid scheduler for execution Command executed: sbatch .command.run Command exit status: 1 Command output: sbatch: error: AssocGrpSubmitJobsLimit sbatch: error: Batch job submission failed: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)

check

projinfo

that your

project

compute allocation code is correct

Thanks, I figured it out and now it started running!

To update Nextflow:

pixi upgrade

and to update

pixi

pixi self-update

Thanks!

Updating Pixi doesn't update the

pixi.lock

or

pixi.toml

. Only package upgrades update those files.

pixi self-update

only upgrades the binary as far as I'm aware.

When the package upgrades update the two files, do they overwrite them or create new files?

overwrites them

Maybe a stupid question, but how do I version control the files? I can rename them into v02 for example but I lose the originial file when it is overwritten?

Ah, by version control I mean by using

git

: See https://nbisweden.github.io/workshop-reproducible-research/home_contents.html

Thanks!