Hi all, I recently used the Airrflow pipeline to do a fairly involved analysis for a collaborator (with many additional downstream steps). As we were typing up the work, I noticed that I had used the defaults for

--reference_fasta

and

--reference_igblast

. I guess that if you don't also set the

--fetch_imgt

flag, the references are downloaded from here and here instead of using the latest IMGT reference data at runtime. Redoing the analysis isn't really feasible, so I'm trying to identify which release of the IMGT database is included with the defaults and have a few questions and suggestions: 1. The imgtdb_base.zip file contains a

IMGT.yaml

file with this information:

source:  <https://www.imgt.org/genedb>
date:    2024.11.26
species:
    - human:Homo+sapiens
    - mouse:Mus

And the igblast_base.zip file contains

/database/*.njs

files with information such as:

{
  "version": "1.2",
  "dbname": "imgt_mouse_tr_d",
  "dbtype": "Nucleotide",
  "db-version": 5,
  "description": "/home/gisela/Projects/Pipelines/test-airrflow/work/19/d57207f9cc332598ed7c76c66dc189/igblast_base/fasta/imgt_mouse_tr_d.fasta",
  "number-of-letters": 60,
  "number-of-sequences": 5,
  "last-updated": "2024-11-26T14:48:00",
  "number-of-volumes": 1,
  "bytes-total": 49924,
  "bytes-to-cache": 300,
  "files": [
    "imgt_mouse_tr_d.ndb",
    "imgt_mouse_tr_d.nhr",
    "imgt_mouse_tr_d.nin",
    "imgt_mouse_tr_d.nog",
    "imgt_mouse_tr_d.nos",
    "imgt_mouse_tr_d.not",
    "imgt_mouse_tr_d.nsq",
    "imgt_mouse_tr_d.ntf",
    "imgt_mouse_tr_d.nto"
  ]
}

As I understand it, none of these versions (e.g.

db-version: 5

) correspond to the IMGT database. Given the date (2024.11.26), should I refer to this page to infer the state of the database at the time these files were prepared? 2. I'm not finding any information about how these files were prepared (e.g., commands used, etc) in the Airrflow documentation, Airrflow report, or Nextflow logs. To ensure reproducibility, I'd recommend making this information more explicit / readily available. 3. Finally, it would seemingly make sense to have

--fetch_imgt

set to True by default. That would ensure that new users, such as myself, use the optimal files by default.

Hi Charles, thank you for your reference related questions and suggestions. I need some help from @Gisela Gabernet who prepared the database to answer your questions. As she's attending a conference this week, I may not have a chance to talk to her until next week. I will reach out to you once I have the answers. We will also consider a way to provide better documentation and tutorial on how to check reference version and how to set or fetch them.

Thank you @Huimin Lyu! I appreciate the help!

About setting a download option from false to true by default is not a really good idea as it would make setting up a pipeline for a secure server like like Lunarc's COSMOS-SENS or Uppmax's Bianca even more annoying.

Yes, @Charles-Alexandre Roy thank you for your message and feedback. Answering your points: 1. You are correct, the date listed on this file is when the IMGT database was downloaded to generate this reference, and in the IMGT log file the version can be inferred. 2. The reference files are generated with the fetch_databases scripts that are in the bin folder of the airrflow pipeline, and it is the scripts that are run when using the

--fetch_imgt

flag. We are currently reworking how references are downloaded and adding support for the AIRR-C reference, so we will make sure to update the documentation accordingly. 3. One reason why this is not the case is due to the API limits that the IMGT server has, which generate issues when trying to download the reference files on the fly each time we run the pipeline. The next airrflow release should also come with updated references, and also the added support for the AIRR-C references.

Thanks for the help @Gisela Gabernet! That all makes sense, and I appreciate all the work you've put into this; it's very impressive!