Hi all, I have successfully tun the pipeline on a single sample with no errors. However, when I try running the pipeline on multiple samples, I am getting the following error (from the nextflow log file):

May-30 16:30:25.150 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task:name=NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size); work dir=/scratch/c.c24016561/bcr_tcr_scrnaseq/bin/tcr/work/28/633dec2a15619caecf4d40473f1da1

  error [nextflow.exception.ProcessFailedException]: Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)` terminated with an error exit status (1)
May-30 16:30:25.195 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)'

Caused by:
  Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)` terminated with an error exit status (1)

Command executed:

  Rscript -e "enchantr::enchantr_report('file_size', \
          report_params=list('input'='all_logs_tabs.txt', 'metadata'='samplesheet.valid.tsv',\
          'outdir'=getwd()))"
  
      cp -r enchantr file_size_report && rm -rf enchantr
  
      echo ""NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE":" > versions.yml
      Rscript -e "cat(paste0('  enchantr: ',packageVersion('enchantr'),'
  '))" >> versions.yml

Command exit status:
  1

Command output:
  1/26                   
  2/26 [global-options]  
  3/26                   
  4/26 [input-parameters]
  5/26 [unnamed-chunk-1] 
  6/26                   
  7/26 [unnamed-chunk-2] 
  8/26 [unnamed-chunk-3] 
  9/26                   
  10/26 [unnamed-chunk-4] 
  11/26                   
  12/26 [unnamed-chunk-5] 

Command error:
  
  
  processing file: _main.Rmd
  
  Quitting from lines  at lines 112-115 [unnamed-chunk-5] (_main.Rmd)
  Error in `bind_rows()`:
  ! Can't combine `..1$output` <character> and `..14$output` <list>.
  Backtrace:
   1. enchantr::getConsoleLogs(log_files)
   2. dplyr::bind_rows(lapply(log_files, enchantr:::formatConsoleLog))
  Warning messages:
  1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr' 
  2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr' 
  3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr' 
  Execution halted

Am I correct in saying this may be a problem with file sizes? I.e. too many samples being processed and therefore the file(s) is too large to run through the enchantr package? Does anyone know how I can get around this? Thanks in advance!!

Hi @Alex Gibbs, can you provide with moreinfo

To help we'd love a few more details please: 1. The Nextflow command you used to launch the pipeline 2. Any custom Nextflow config files 3. The full error message 4. The corresponding

.nextflow.log

file, where you are running the analysis. Remember to use triple backslashes on their own lines for code blocks. Also take a look at https://nf-co.re/usage/troubleshooting for more information on debugging failed pipelines. Thanks!

I don’t think it is due to too many samples, I’ve tested the pipeline with ~200 samples, but I think it might be due to one of the samples being empty or something of the sort because it says one is of type character and the other of type list. I will try and solve the bug. Which type of data were you analyzing? Assembled reads in AIRR format? And TCR data?

Thanks for getting back to me @Gisela Gabernet, and apologies for the delayed reply. I will attach the .YAML, the .config, and the head of the samplesheet.tsv file below along with the command used to launch the pipeline. I have tried running this pipeline using both BCR and TCR raw sequencing data and am having exactly the same error message. I am wondering if there may be a problem on my side, so have sent our computing team an email to see if they can find the cause. TCR example: bcr-airrflow-params.yaml

mode: fastq
input: ./bcr-samplesheet.tsv
library_generation_method: sc_10x_genomics
reference_10x: /scratch/c.c123456/data.other/airrflow-test/resources/refdata-cellranger-vdj-GRCh38-alts-ensembl-7.1.0.tar.gz
clonal_threshold: 0
outdir: ../output
email: myemail@cardiff.ac.uk

ag.config

params {
  config_profile_description = 'Super Computing Wales'
  config_profile_contact = 'myemail@cardif.ac.uk'
  config_profile_url = '<https://supercomputing.wales/>'
}
singularity {
  enabled = true
  autoMounts = true
}
executor {
  name = 'slurm'
  queueSize = 10
  queue = 'htc'
}
params {
  max_memory = 180.GB
  max_cpus = 20
  max_time = 72.h
}

process {
 beforeScript = 'module load singularity-ce/3.11.4'
 clusterOptions = '--account=scw1234'
}

head(bcr-samplesheet.tsv) - this file contains 8 samples

sample_id	filename_R1	filename_R2	subject_id	species	tissue	pcr_target_locus	single_cell	sex	age	biomaterial_provider
B2-SC17T013_H2WFVDSXC	/scratch/c.c123456/data.other/airrflow-test/input/BCR/B2-SCI7T013_H2WFVDSXC_S2_L002_R1_001.fastq.gz	/scratch/c.c123456/data.other/airrflow-
test/input/BCR/B2-SCI7T013_H2WFVDSXC_S2_L002_R2_001.fastq.gz	Subject02	human	PBMC	IG	FALSE	M	NA	10X Genomics

command to launch pipe:

nextflow run nf-core/airrflow -r dev -params-file bcr-airrflow-params.yaml -profile singularity -c ~/ag.config

Thanks I think it is a bug from Airrflow in generating the report somewhere. Could you try whether this has been solved in the latest release?

nextflow run nf-core/airrflow -r 4.1.0

Thanks, @Gisela Gabernet. I will try both of these and let you know how it goes. Appreciate you looking into this. No massive rush, I can run these individually for now as I only have a small number of samples.

Hi @Gisela Gabernet, did this issue ever get solved? I am getting the same error message (running -r 4.2.0).

Hi @Melanie Kirch, we tried investigating this error but it was hard without having the example data for which this fails. Could you share the exact error message you are getting? And if possible, all of the files in the work directory of the process that fails?

Hi @Gisela Gabernet, did this issue ever get solved? I am getting the same error message (running -r 5.0.0). And my error message is as follows:

ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)'

Caused by:
  Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)` terminated with an error exit status (1)


Command executed:

  Rscript -e "enchantr::enchantr_report('file_size', \
          report_params=list('input'='all_logs_tabs.txt', 'metadata'='validated_input.tsv',\
          'outdir'=getwd()))"
  
      cp -r enchantr file_size_report && rm -rf enchantr
  
      echo ""NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE":" > versions.yml
      Rscript -e "cat(paste0('  enchantr: ',packageVersion('enchantr'),'
  '))" >> versions.yml

Command exit status:
  1

Command output:
  1/26                   
  2/26 [global-options]  
  3/26                   
  4/26 [input-parameters]
  5/26 [unnamed-chunk-1] 
  6/26                   
  7/26 [unnamed-chunk-2] 
  8/26 [unnamed-chunk-3] 
  9/26                   
  10/26 [unnamed-chunk-4] 
  11/26                   
  12/26 [unnamed-chunk-5] 

Command error:
  
  
  processing file: _main.Rmd
  1/26                   
  2/26 [global-options]  
  3/26                   
  4/26 [input-parameters]
  5/26 [unnamed-chunk-1] 
  6/26                   
  7/26 [unnamed-chunk-2] 
  8/26 [unnamed-chunk-3] 
  9/26                   
  10/26 [unnamed-chunk-4] 
  11/26                   
  12/26 [unnamed-chunk-5] 
  Error in `consoleLogsAsGraphs()`:
  ! Expecting one row with metadata for this component.
  Backtrace:
      x
   1. \-enchantr::consoleLogsAsGraphs(log_data, metadata)
  Warning messages:
  1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr' 
  2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr' 
  3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr' 
  
  Quitting from _main.Rmd:118-122 [unnamed-chunk-5]
  Execution halted

Work dir:
  /data2/data2_mailab003/project/Proj_NPC_Tislelizumab/02.CleanData/02_nfcore_pipeline/BCR/workdir/4f/6d38d727a1243c354eb7cba73e7c88

Container:
  <http://docker.io/immcantation/airrflow:5.0.0|docker.io/immcantation/airrflow:5.0.0>

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details

Hi @Huimin Lyu and @ssnn have been working on a solution to this issue that will be available in the following release. We will let you know as soon as this is available in the airrflow dev branch (should be there in the next couple of weeks) and you could test whether the issue has been solved for your data.

YES. We already have a solution but it's not in the dev branch yet. As I will attend a conference this week, I am not able to implement the solution right away. I plan to test it and add it to airrflow dev branch next week. I will contact you as soon as it's available and tell you how to test it with your data. @Suchen Li

Oh, thanks! I just noticed that you’ve already replied to that issue. @Huimin Lyu

Hi, @Suchen Li We are working on that. We just noticed some issues, so we are fixing the bugs. Actually, if you are interested in trying the new solutions, you can run the pipeline from the branch (novel-allele-and-genotype) we are working on (usually we do not recommend our user to do that because it's not a stable version. Even the user gets good results, it's impossible to publish the results from an unfixed nf-core/airrflow version). I do not want to waste your time, however, if you want to test whether it works in advance, you can run the pipeline with command

nextflow run nf-core/airrflow -r novel-allele-and-genotype

, the rest should be the same as your previous command.