Hi All! I think I need some help/advice on how to best use airrflow with my custom single-cell AIRR-seq data. I would like to start the analysis from the fastq files, however from airrflow's documentation, as I understood, only libraries generated with 10X method are supported for single-cell AIRR-seq starting from fastq. However, I also saw that TRUST4 is implemented in the pipeline, but from the graph below it looks like within the pipeline TRUST4 is intended for scRNA-seq data, not AIRR-seq. I wonder if it is possible, as a workaround in my case, to run airrflow with TRUST4 with adjusting its parameters to match the format of my data, which would be:

run-trust4 -f hg38_bcrtcr.fa --ref human_IMGT+C.fa -u $R2 --barcode $R1 --UMI $R1 --readFormat 'bc:12:21,um:0:11' --barcodeWhitelist whitelist.tsv

Would you say this workaround is possible, or are there more things to consider? Thank you in advance for thinking along 🙌

Hi Sofia, just checkin your command it would in principle almost be possible to analyze your data with TRUST4, the only issue is that so far we don't give the option to provide a `barcodeWhitelist`:

nextflow run nf-core/airrflow -r 4.2.0 \
-profile docker \
--mode fastq \
--library_generation_method trust4 \
--cell_barcode_read R1 \
--umi_read R1 \
--read_format "bc:12:21,um:0:11"

you could give the

barcodeWhitelist

in your own config for now. but we can also add it as a feature to the nf-core module I agree

I think it's not possible to pass files in the config file if that hasn't been implemented into the module. Or is there an input channel for the barcodeWhitelist that I missed?

oh ok didnt know that I did a pr to change the module, then we can do a pr to airrflow to add the input https://github.com/nf-core/modules/pull/8368

sounds good, I've just reviewed!

Thank you so much @Gisela Gabernet and @Mark Polster for picking it up so quickly! Indeed, the data has 384 custom cell barcodes, hence would be great to have whitelist as additional input.

@Sofia Drobysheva I'm still not sure whether trust4 will work with your scVDJ data though, as you mentioned it is intended for scRNAseq in principle. I would give it a try to run trust4 outside of the pipeline first to see whether this works for you.

@Gisela Gabernet thank you for the heads up! I ran trust4 on our scVDJ data before and, as far as I can tell, got good results back. Airrflow also includes fastp preprocessing step, which I have not applied to my data prior to running trust4 in the past. Hence, one of my curiosity points is to see how fastp affects trust4 outputs. And the main interest, of course, is to see which outputs I will get from the clonotype and repertoire analysis steps 🙂

Sounds good!

Hi Sofia and Gisela I added the barcodewhitelist option now to airrflow. Sofia, it would be awesome if you could test it quickly since I dont have sufficient test data on the hand right now. Basically I just added the parameter so it shouldnt have changed. You can start airrflow from this PR https://github.com/nf-core/airrflow/pull/385 and then add the

--trust4_barcode_whitelist

parameter. Make sure to add the

--trust4_...

before each trust4 param. We also integrated this in this PR

Thank you @Mark Polster so much for a quick implementation! I will do my best to test it tomorrow, will follow up!

great, thanks for helping with the debugging let me know if you have any troubles running the revision 🙂

@Mark Polster I left a comment on the PR, but basically the barcodeList param needs to be read as part of airrflow still inside a channel and passed to the process for it to work.

oh my god. You shouldn’t do such things in between meetings. I’ll fix it tomorrow

Hi! To follow up on the progress so far, I have cloned the modified airrlow code locally and run the following command:

nextflow run <http://main.nf|main.nf> \
-profile docker \
--mode fastq \
--library_generation_method trust4 \
--trust4_cell_barcode_read R1 \
--trust4_umi_read R1 \
--trust4_read_format "bc:12:21,um:0:11" \
--trust4_barcode_whitelist whitelist.tsv \
--input samplesheet.tsv \
--outdir test_airr

As I run the pipeline locally, I have limited resources and get the following error:

ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:DATABASES:UNZIP_REFERENCE_FASTA (unzip_db)'

Caused by:
  Process requirement exceeds available memory -- req: 36 GB; avail: 16 GB

I tried providing a nextflow.config that looks like this:

params {
    max_cpus   = 2
    max_memory = 16.GB
    max_time   = 6.h
}

However the parameters do not seem to be picked up by the pipeline, as there is also a warning saying:

WARN: The following invalid input values have been detected:

* --max_cpus: 2
* --max_memory: 16 GB
* --max_time: 6h

I will look for a workaround, potentially running airrlow on HPC (currently I am running the code locally for convenience of using docker)

Hi Sofia, your config should look sth like this:

process {
  withName: UNZIP_REFERENCE_FASTA {
    memory = 16.GB
    cpus = 2
    time = 6h 
  }
}

you can also see here https://nf-co.re/docs/usage/getting_started/configuration#tuning-workflow-resources for further infos

Hi Mark, you are right, now its running again, thanks! Small note, I think in your example, it should be

time = 6.h

instead of

time = 6h

, then it works as a charm 🙂

glad it works 🙂 thanks for pointing this out, I was just writing this from head and wasnt sure about time :D

The config Mark provided affects only one process, if you want to restrict the memory for all processes you can use this one:

process {
    resourceLimits = [ memory: 16.GB, time: 24.h, cpus: 8 ]
}

Awesome! no I didn’t know this either