Hi all, I’m running into repeated failures in the PARSE_LOGS step of nf-core/airrflow and wanted to check whether this is a known issue or whether something in my setup might be triggering it. Setup:

$VSC_HOME/nextflow run nf-core/airrflow -r 4.3.0 \
  -config /censored/slurm_scripts/my_no_cleanup.config \
  -profile vsc_calcua,single_node,nebnext_umi_bcr \
  --input /censored/samplesheets/10_IGH.tsv \
  --outdir /censored/10/IGH/ \
  --lineage_trees TRUE \
  --clonal_threshold 0.12 \
  -w ${NXF_WORK} \
  -resume

What happens Most of the pipeline finishes normally, but at the end the process

REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS

fails with:

ValueError: arrays must all be same length

from

log_parsing.py

when it tries to build the final reporting DataFrame. The error occurs consistently for some samples but not others. Context • I originally sequenced these samples once and processed them without issues (only the collapseDuplicates() fail that I reported on GitHub). • Later, I sequenced the same samples again and merged the two FASTQ files _by simple concatenation_: •

cat sample_run1.fastq.gz sample_run2.fastq.gz > sample.fastq.gz

• The two sequencing runs had different FASTQ headers. • The merged FASTQ kept the filename of the first run. • The workflow and slurm scripts are the same as the first pipeline run, except that I additionally set

process.cleanup = true

in an optional config. • Only some merged samples trigger the PARSE_LOGS error; others complete normally. What I observe in the failing runs • The rest of the pipeline (UMI processing, Presto steps, CHANGEO DB creation, etc.) completes successfully. • The failure happens only in the summary/log-reporting stage. • When PARSE_LOGS runs, other samples still appear to be in progress or cached. I’m not sure whether this behavior is expected when FASTQs from two runs are merged, or whether PARSE_LOGS might be sensitive to timing or partial log output. If helpful, I can share example

_command_log

files or a minimal reproducer. Let me know what would be most useful.

executor >  local (150)
[6f/70172d] NFC…S:UNZIP_IGBLAST (unzip_db) | 1 of 1 ✔
[fa/d7bb95] NFC…REFERENCE_FASTA (unzip_db) | 1 of 1 ✔
[01/7b4bbf] NFC…LESHEET_CHECK (10_IGH.tsv) | 1 of 1 ✔
[-        ] NFC…ASTQ_INPUT_CHECK:CAT_FASTQ -
[65/c7946e] NFC…Y:PRESTO_UMI:FASTP (s10_3) | 6 of 6 ✔
[b9/89d403] NFC…I:RENAME_FASTQ_UMI (s10_3) | 6 of 6 ✔
[26/6a1e8b] NFC…TO_UMI:GUNZIP_UMI (s10_85) | 6 of 6 ✔
[5c/615d35] NFC…STO_FILTERSEQ_UMI (s10_85) | 6 of 6 ✔
[c4/6cd8bd] NFC…MERS_SCORE_UMI_R1 (s10_85) | 6 of 6 ✔
[e5/96da44] NFC…MERS_SCORE_UMI_R2 (s10_-5) | 6 of 6 ✔
[a8/3a0e74] NFC…RESTO_PAIRSEQ_UMI (s10_85) | 6 of 6 ✔
[06/4cfb04] NFC…UILDCONSENSUS_UMI (s10_-5) | 6 of 6 ✔
[52/d08026] NFC…ENSUS_PAIRSEQ_UMI (s10_-5) | 6 of 6 ✔
[f4/b7b448] NFC…EPAIRS_SEQUENTIAL (s10_85) | 6 of 6 ✔
[c7/841fb2] NFC…STO_ALIGN_CREGION (s10_85) | 6 of 6 ✔
[c8/fa0b96] NFC…_POSTASSEMBLY_UMI (s10_85) | 6 of 6 ✔
[fa/404aed] NFC…DERS_COLLAPSE_UMI (s10_85) | 6 of 6 ✔
[7f/9a7028] NFC…ADERS_PRIMERS_UMI (s10_85) | 6 of 6 ✔
[d8/e7bd7e] NFC…LLAPSESEQ_CREGION (s10_85) | 6 of 6 ✔
[6c/103e03] NFC…DERS_METADATA_UMI (s10_85) | 6 of 6 ✔
[6c/d681c8] NFC…ESTO_SPLITSEQ_UMI (s10_85) | 6 of 6 ✔
[51/d18149] NFC…ANGEO_ASSIGNGENES (s10_85) | 6 of 6, retries: 2 ✔
[82/8349d1] NFC…ON:CHANGEO_MAKEDB (s10_85) | 6 of 6 ✔
[4e/d394e6] NFC…ON:FILTER_QUALITY (s10_85) | 5 of 6
[e1/bbcfdc] NFC…GEO_PARSEDB_SPLIT (s10_-5) | 5 of 5
[13/930ae0] NFC…TER_JUNCTION_MOD3 (s10_-5) | 5 of 5
[4e/806aa0] NFC…N:ADD_META_TO_TAB (s10_-5) | 5 of 5
[76/3e1c8b] NFC…LLAPSE_DUPLICATES (s10_14) | 5 of 5, retries: 1
[02/9c97b5] NFC…EPORTING:PARSE_LOGS (logs) | 0 of 1, retries: 3 ✘
Plus 8 more processes waiting for tasks…
[b5/d5e8bd] NOTE: Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS (logs)` terminated with an error exit status (1) -- Execution is retried (1)
[5c/75f16e] NOTE: Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS (logs)` terminated with an error exit status (1) -- Execution is retried (2)
[68/6ba961] NOTE: Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS (logs)` terminated with an error exit status (1) -- Execution is retried (3)
ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS (logs)'

Caused by:
  Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS (logs)` terminated with an error exit status (1)


Command executed:

  log_parsing.py 
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:PARSE_LOGS":
      python: $( echo $(python --version | grep -o "[0-9\. ]\+") )
      pandas: $(echo $(python -c "import pkg_resources; print(pkg_resources.get_distribution('pandas').version)"))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Traceback (most recent call last):
    File "/censored/.nextflow/10/assets/nf-core/airrflow/bin/log_parsing.py", line 189, in <module>
      df_process = pd.DataFrame.from_dict(
    File "/usr/local/lib/python3.9/site-packages/pandas/core/frame.py", line 1309, in from_dict
      return cls(data, index=index, columns=columns, dtype=dtype)
    File "/usr/local/lib/python3.9/site-packages/pandas/core/frame.py", line 468, in __init__
      mgr = init_dict(data, index, columns, dtype=dtype)
    File "/usr/local/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 283, in init_dict
      return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
    File "/usr/local/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 78, in arrays_to_mgr
      index = extract_index(arrays)
    File "/usr/local/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 397, in extract_index
      raise ValueError("arrays must all be same length")
  ValueError: arrays must all be same length

Work dir:
  /censored/10/work/02/9c97b57257e250e4a3f0c88cca72b3

Container:
  /censored/singularity/cache/depot.galaxyproject.org-singularity-pandas-1.1.5.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details
-[nf-core/airrflow] Pipeline completed with errors-
WARN: Killing running tasks (1)

Thanks @Thomas-Wolf Verdonckt for reporting. We’ve fixed several bugs in the PARSE_LOGS process lately. Could you test whether you’re still experiencing issues with the

-r dev

pipeline version that is about to be released?

Hi @Gisela Gabernet, running with

-r dev

did indeed solve the PARSE_LOGS issue. Has there been any changes made between 4.4.0 and 4.3.0 in how the repertoires are assembled? In other words, can I directly compare the repertoires assembled with

-r dev

to the ones I already have or should I re-assemble all repertoires with this latest pipeline version?

There has been only one major change: we’ve updated pRESTO to v.0.7.6. The previous v0.7.4 of pRESTO had an overly conservative computing of average read quality (using the average of the probabilities) which resulted in filtering out quite some sequences due to low quality. This has been fixed in v0.7.6 using the average of the log probabilities, which is also what pRESTO versions <0.7.1 were using. So you will likely recover more sequences with this newer version, depending on your protocol and sequencing read quality. Other changes are small bugfixes and can be seen in the changelog.

Thank you for the update. That sounds like a large enough difference to justify re-running all assemblies. Do you have any visibility into whether there will be significant changes to dev before 4.4.0 is released? I would like to proceed with my analysis, but I would prefer not to have to cite a dev version in my Materials and Methods section. If the upcoming changes are limited, I can run the assemblies now. and cite 4.4.0

There will not be major changes any more. I’m planning to release latest beginning of next week though