Hello everyone, I'm back in this channel, as I've encountered a brand new error. But first, I want to thank Mark for helping me see more clearly my previous problem. But I still need help on this project. This time, all the steps worked except the last one :

NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)

There seems to be a problem related to the size of the final output file, but I'm not sure. Please note : The error repeated several times, but did not immediately interrupt execution. I'm continuing my research in parallel, but if anyone has any ideas or has already encountered this situation, I'd love to hear from you ! Error that cause the code to stop :

[59/77eb5c] NFCORE_AIRRFLOW:AIRRFLOW:DATABASES:UNZIP_IGBLAST (unzip_db)                                      [100%] 1 of 1, cached: 1 ✔
[6a/cc472a] NFCORE_AIRRFLOW:AIRRFLOW:DATABASES:UNZIP_REFERENCE_FASTA (unzip_db)                              [100%] 1 of 1, cached: 1 ✔
[1a/d5604f] NFC…RRFLOW:AIRRFLOW:RNASEQ_INPUT:FASTQ_INPUT_CHECK:SAMPLESHEET_CHECK (assembled_samplesheet.tsv) [100%] 1 of 1, cached: 1 ✔
[0e/e0172a] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:FASTP (BLOD_AM1_2_A1)                                      [100%] 2 of 2, cached: 2 ✔
[8d/9b4a0b] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:RENAME_FASTQ_TRUST4 (BLOD_AM1_1_A1)                        [100%] 2 of 2, cached: 2 ✔
[43/de180c] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:PREPARE_TRUST4_REFERENCE (BLOD_AM1_2_A1)                   [100%] 1 of 1, cached: 1 ✔
[46/70d4d5] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:TRUST4 (BLOD_AM1_1_A1)                                     [100%] 2 of 2 ✔
[c0/918f42] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:CHANGEO_PARSEDB_SELECT_LOCUS (BLOD_AM1_1_A1)               [100%] 2 of 2 ✔
[1e/c63c82] NFCORE_AIRRFLOW:AIRRFLOW:RNASEQ_INPUT:CHANGEO_CONVERTDB_FASTA_FROM_AIRR (BLOD_AM1_1_A1)          [100%] 2 of 2 ✔
[7a/0f5f51] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:CHANGEO_ASSIGNGENES (BLOD_AM1_1_A1)                      [100%] 2 of 2 ✔
[0d/fce6fe] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:CHANGEO_MAKEDB (BLOD_AM1_1_A1)                           [100%] 2 of 2 ✔
[a3/8932d8] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:FILTER_QUALITY (BLOD_AM1_1_A1)                           [100%] 2 of 2 ✔
[24/d08b1d] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:CHANGEO_PARSEDB_SPLIT (BLOD_AM1_1_A1)                    [100%] 2 of 2 ✔
[0f/9cf394] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:FILTER_JUNCTION_MOD3 (BLOD_AM1_1_A1)                     [100%] 2 of 2 ✔
[60/d82c51] NFCORE_AIRRFLOW:AIRRFLOW:VDJ_ANNOTATION:ADD_META_TO_TAB (BLOD_AM1_1_A1)                          [100%] 2 of 2 ✔
[66/68c186] NFCORE_AIRRFLOW:AIRRFLOW:BULK_QC_AND_FILTER:COLLAPSE_DUPLICATES (BLOD_AM1_1_A1)                  [100%] 2 of 2 ✔
[-        ] NFCORE_AIRRFLOW:AIRRFLOW:SINGLE_CELL_QC_AND_FILTERING:SINGLE_CELL_QC                             -
[cf/275be0] NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:REPORT_THRESHOLD (all_reps)                             [100%] 1 of 1 ✔
[61/e8a4ae] NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:DEFINE_CLONES_COMPUTE (AM1_2_A1)                        [100%] 2 of 2 ✔
[-        ] NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:DEFINE_CLONES_REPORT                                    -
[d2/cbb5b8] NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)              [100%] 1 of 1, failed: 1 ✘
[-        ] NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:AIRRFLOW_REPORT                           [  0%] 0 of 1
[-        ] NFCORE_AIRRFLOW:AIRRFLOW:MULTIQC                                                                 -
-[nf-core/airrflow] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)'

Caused by:
  Process `NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)` terminated with an error exit status (1)


Command executed:

  Rscript -e "enchantr::enchantr_report('file_size', \
          report_params=list('input'='all_logs_tabs.txt', 'metadata'='samplesheet.valid.tsv',\
          'outdir'=getwd()))"
 
      cp -r enchantr file_size_report && rm -rf enchantr
 
      echo ""NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE":" > versions.yml
      Rscript -e "cat(paste0('  enchantr: ',packageVersion('enchantr'),'
  '))" >> versions.yml

Command exit status:
  1

Command output:
  1/26                   
  2/26 [global-options]  
  3/26                   
  4/26 [input-parameters]
  5/26 [unnamed-chunk-1]
  6/26                   
  7/26 [unnamed-chunk-2]
  8/26 [unnamed-chunk-3]
  9/26                   
  10/26 [unnamed-chunk-4]
  11/26                   
  12/26 [unnamed-chunk-5]

Command error:
  1/26                   
  2/26 [global-options]  
  3/26                   
  4/26 [input-parameters]
  5/26 [unnamed-chunk-1]
  6/26                   
  7/26 [unnamed-chunk-2]
  8/26 [unnamed-chunk-3]
  9/26                   
  10/26 [unnamed-chunk-4]
  11/26                   
  12/26 [unnamed-chunk-5]
  Error in `select()`:
  ! Can't select columns that don't exist.
  x Column `output` doesn't exist.
  Backtrace:
       x
    1. +-enchantr::getConsoleLogs(log_files)
    2. | +-dplyr::bind_rows(lapply(log_files, enchantr:::formatConsoleLog))
    3. | | \-rlang::list2(...)
    4. | \-base::lapply(log_files, enchantr:::formatConsoleLog)
    5. |   \-enchantr (local) FUN(X[[i]], ...)
    6. |     \-... %>% as.data.frame(stringsAsFactors = F)
    7. +-base::as.data.frame(., stringsAsFactors = F)
    8. +-dplyr::select(., input, output, task, input_size, output_size)
    9. +-dplyr:::select.data.frame(., input, output, task, input_size, output_size)
   10. | \-tidyselect::eval_select(expr(c(...)), data = .data, error_call = error_call)
   11. |   \-tidyselect:::eval_select_impl(...)
   12. |     +-tidyselect:::with_subscript_errors(...)
   13. |     | \-base::withCallingHandlers(...)
   14. |     \-tidyselect:::vars_select_eval(...)
   15. |       \-tidyselect:::walk_data_tree(expr, data_mask, context_mask)
   16. |         \-tidyselect:::eval_c(expr, data_mask, context_mask)
   17. |           \-tidyselect:::reduce_sels(node, data_mask, context_mask, init = init)
   18. |             \-tidyselect:::walk_data_tree(new, data_mask, context_mask)
   19. |               \-tidyselect:::as_indices_sel_impl(...)
   20. |                 \-tidyselect:::as_indices_impl(...)
   21. |                   \-tidyselect:::chr_as_locations(x, vars, call = call, arg = arg)
   22. |                     \-vctrs::vec_as_location(...)
   23. \-vctrs (local) `<fn>`()
   24.   \-vctrs:::stop_subscript_oob(...)
   25.     \-vctrs:::stop_subscript(...)
   26.       \-rlang::abort(...)
  Warning messages:
  1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr'
  2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr'
  3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr'
 
  Quitting from _main.Rmd:111-115 [unnamed-chunk-5]
  Execution halted

Work dir:
  /mnt/DOSI/TDLAB/BIOINFO/Project/BLOD/BLOD_A1/01_Reference/02_Airrflow/work/d2/cbb5b89261cb0505797be00ed5952e

Container:
  /mnt/DOSI/TDLAB/BIOINFO/Project/BLOD/BLOD_A1/01_Reference/02_Airrflow/work/singularity/docker.io-immcantation-airrflow-4.3.0.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details

Thank you in advance

Hi @Drystan, we’d like to help debugging the issue. Could you also share the command you used to run the pipeline and your samplesheet?

Hi @Gisela Gabernet, Thank you very much for your help and for the suggestions, I will test it. I’ll try to share the data you requested. I hope you received everything you needed. So, here is the command (the .sh) I used to run the pipeline :

nextflow run nf-core/airrflow -r 4.3.0 \
-profile singularity \
--mode fastq \
--input assembled_samplesheet.tsv \
--library_generation_method trust4 \
--clonal_threshold 0.1 \
-c resource.config \
--outdir /mnt/DOSI/TDLAB/BIOINFO/Project/BLOD/BLOD_A1/05_Output/01_FlashPipe/02_Airrflow/ \
-resume

I’ve also attached the samplesheet

.tsv

file used in the run and the zip of the directory

/mnt/DOSI/TDLAB/BIOINFO/Project/BLOD/BLOD_A1/01_Reference/02_Airrflow/work/d2/cbb5b89261cb0505797be00ed5952e/

. Thanks again, Drystan

Thank you @Drystan, this will be very helpful in debugging

Hello @Gisela Gabernet, here what you asked for. I hope it will help you this time. Let me know if this isn't what you expected, or if you need more files.

Thank you @Drystan, this time the log files seem to be all there. We will debug this and let you know 🙂

Hello, sorry for only responding now, but due to various circumstances, I have not been able to access my computer and therefore the data. I will try to send you what you except to help you. But I will have to check if I have permission to send them to you. I will get back to you shortly. Have a nice day.

Thank you very much @Drystan!

Hi @Gisela Gabernet, thanks for all your work on the pipeline! Is there already an update on this issue? I have run into the same error. Thanks in advance!

Hi @Carlotta Schieler, this issue should be fixed in the airrflow

dev

version, which will be released very soon (hopefully next week). Could you test whether specifying

-r dev

solves the issue for you?

Hi @Gisela Gabernet, I tried it now with

-r dev

and it used

4.4dev

, but I still have the same error: `[57/8f646a] NOTE: Process

NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:REPORT_FILE_SIZE (file_size)

terminated with an error exit status (1) -- Error is ignored [11/4dac6d] NOTE: Process

NFCORE_AIRRFLOW:AIRRFLOW:REPERTOIRE_ANALYSIS_REPORTING:AIRRFLOW_REPORT (all_reps)

terminated with an error exit status (1) -- Error is ignored` I am happy to share log file etc. if that helps. The only warn message I get in the beginning is

WARN: Found the following unidentified headers in samplesheet.tsv

and I am running the pipeline directly from my airr_rearrangment files with

--mode assembled

.

Hi Carlotta, thank you, if you could share the full content (following symlinks) of the work directory for this process that would enable us to debug the error.

Hi @Gisela Gabernet, thanks for you answer. Do you need the full work directory for debugging? Looking through it, there is a lot of sensitive data I cannot easily share.

We would need at least the full error message to start with debugging. If you go to your work directory, under the subdirectory

57/8f646a

that starts with these numbers, you'll find a hidden file (need to list with

ls -la

) with name

.command.err

that contains the error message. If this is empty, sometimes the error goes to the

.command.log

or

.command.out

file instead.

I had reran the pipeline a couple more times, but always led to the same error, so I guess this error message from the last run is still the same:

cat .command.err


processing file: _main.Rmd
Error in `bind_rows()`:
! Can't combine `log_1$output` <character> and `log_69$output` <list>.
Backtrace:
     x
  1. +-enchantr::readConsoleLogs(log_files)
  2. | \-dplyr::bind_rows(logs_list, .id = "log_id")
  3. |   \-vctrs::vec_rbind(!!!dots, .names_to = .id, .error_call = current_env())
  4. \-vctrs (local) `<fn>`()
  5.   \-vctrs::vec_default_ptype2(...)
  6.     +-base::withRestarts(...)
  7.     | \-base (local) withOneRestart(expr, restarts[[1L]])
  8.     |   \-base (local) doWithOneRestart(return(expr), restart)
  9.     \-vctrs::stop_incompatible_type(...)
 10.       \-vctrs:::stop_incompatible(...)
 11.         \-vctrs:::stop_vctrs(...)
 12.           \-rlang::abort(message, class = c(class, "vctrs_error"), ..., call = call)
Warning messages:
1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr' 
2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr' 
3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr' 

Quitting from _main.Rmd:86-89 [unnamed-chunk-2]
Execution halted

Thank you, yes this is slightly different from the the error from @Drystan above so we'll have a look

Thank you!