https://nf-co.re logo
Join Slack
Powered by
Hi, sorry for yet another question. The pipeline ...
# airrflow
j
Hi, sorry for yet another question. The pipeline has been running fine on our HPC, but has failed at the: 
NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:FIND_CLONAL_THRESHOLD
task.
Copy code
Command exit status:
  1

Command output:
  1/36                       
  2/36 [global-options]      
  3/36                       
  4/36 [input-parameters]    
  5/36 [unnamed-chunk-1]     
  6/36                       
  7/36 [unnamed-chunk-2]     
  8/36                       
  9/36 [unnamed-chunk-3]     
  10/36                       
  11/36 [unnamed-chunk-4]     
  12/36 [unnamed-chunk-5]     
  13/36                       
  14/36 [sum_heavy_0]         
  15/36                       
  16/36 [distToNearest_bulk]  

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  
  
  processing file: _main.Rmd
  Error in `unserialize()`:
  ! error reading from connection

Backtrace:
    1. bookdown::render_book(...)
    2. bookdown:::render_cur_session(...)
    3. rmarkdown::render(main, output_format, ..., clean = clean, envir = envir)
    4. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
    5. knitr:::process_file(text, output)
       ...
   16. base::withRestarts(...)
   17. base (local) withRestartList(expr, restarts)
   18. base (local) withOneRestart(withRestartList(expr, restarts[-nr]), restarts[[nr]])
   19. base (local) docall(restart$handler, restartArgs)
   21. evaluate (local) fun(base::quote(`<smplErrr>`))
  Warning messages:
  1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr' 
  2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr' 
  3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr' 
  4: In normalizePath(file.path(report_params[["outdir"]], "assets",  :
    path[1]="assets/logo.png": No such file or directory
  5: In file.copy(report_params[["logo"]], target_logo, recursive = T,  :
    'recursive' will be ignored as 'to' is not a single existing directory
  6: In file.create(to[okay]) :
    cannot create file 'assets/logo.png', reason 'No such file or directory'
  
  Quitting from lines 221-243 [distToNearest_bulk] (_main.Rmd)
  Execution halted

Execution cancelled -- Finishing pending tasks before exit
Automatic clone_threshold is 'NA'. Consider setting --clonal_threshold manually.
Does anyone have any ideas what the issue might be?
s
I am not sure what the actual error is, if an issue copying the logo for the report (unlikely?) or just that the threshold can not be determined automatically. The threshold one is easier to debug, so let's start here. Usually, the error message "Automatic clone_threshold is 'NA'. Consider setting --clonal_threshold manually" happens when airrflow can't find a threshold automatically, and then you need to provide one manually, and resume work. In your output directory, you should have the folder 
clonal_analysis/find_threshold/all_reps_dist_report/
with a report that has figures of the distance distributions. Do you have this folder and report? If so, take a look at the plots, and select a threshold that makes sense for your data. For BCRs, something around 0.1-0.15 is common. Then you can set 
--clonal_threshold 0.15
(or the value you choose) and `-resume`to continue with the analysis using the selected threshold.
j
Hi, thanks. I don’t have the 
clonal_analysis
folder, it seems to have had an error in the 
distToNearest
command but I can’t work out what the issue is. I’ll try re-running with with a fixed threshold. Can I edit the nf-core command with resend to pick up where it left off?
Sorry, I am still struggling with the error, I have run the airrflow pipeline with the following command:
Copy code
nextflow run nf-core/airrflow -bg -r 4.2.0 
-profile clontech_umi_bcr,cambridge 
--partition sapphire 
--project PROJECT-CODE 
--input /rds/HvBBlpwx4dc/data/nf_core_airr_sample_table.tsv 
-resume 
--outdir rds/HvBBlpwx4dc/data/nf_airr/nf_airr_20250210 
--clonal_threshold 0.13 
--lineage_trees true 
--multiqc_title Nf-AIRR_BULK_BCR_Run 
--email email_Address 
-with-report /rds/HvBBlpwx4dc/data/nf_airr/nf_airr_20250210/report.html
This has been processing the files fine, then fails on 
ERROR ~ Error executing process > 'NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:REPORT_THRESHOLD (all_reps)
Which has the following error message:
Copy code
Caused by:
  Process `NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:REPORT_THRESHOLD (all_reps)` terminated with an error exit status (1)


Command executed:

  Rscript -e "enchantr::enchantr_report('find_threshold', \
          report_params=list('input'='find_threshold_samplesheet.txt',\
              'cloneby'='subject_id',\
              'crossby'='subject_id',\
              'singlecell'='single_cell',\
              'outdir'=getwd(),\
              'nproc'=16,\
              'outname'='all_reps',\
              'log'='all_reps_threshold_command_log',\
              'logo'='nf-core-airrflow_logo_reports.png' ,'findthreshold_cutoff'='user','findthreshold_edge'=0.9,'findthreshold_method'='gmm','findthreshold_model'='gamma-norm','findthreshold_spc'=0.995,'subsample'=10000))"
  
      cp -r enchantr all_reps_dist_report && rm -rf enchantr
  
      echo "NFCORE_AIRRFLOW:AIRRFLOW:CLONAL_ANALYSIS:REPORT_THRESHOLD": > versions.yml
      Rscript -e "cat(paste0('  enchantr: ',packageVersion('enchantr'),'
  '))" >> versions.yml

Command exit status:
  1

Command output:
  1/36                       
  2/36 [global-options]      
  3/36                       
  4/36 [input-parameters]    
  5/36 [unnamed-chunk-1]     
  6/36                       
  7/36 [unnamed-chunk-2]     
  8/36                       
  9/36 [unnamed-chunk-3]     
  10/36                       
  11/36 [unnamed-chunk-4]     
  12/36 [unnamed-chunk-5]     
  13/36                       
  14/36 [sum_heavy_0]         
  15/36                       
  16/36 [distToNearest_bulk]  

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  
  
  processing file: _main.Rmd
  Error in `unserialize()`:
  ! error reading from connection
  Backtrace:
    1. bookdown::render_book(...)
    2. bookdown:::render_cur_session(...)
    3. rmarkdown::render(main, output_format, ..., clean = clean, envir = envir)
    4. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
    5. knitr:::process_file(text, output)
       ...
   16. base::withRestarts(...)
   17. base (local) withRestartList(expr, restarts)
   18. base (local) withOneRestart(withRestartList(expr, restarts[-nr]), restarts[[nr]])
   19. base (local) docall(restart$handler, restartArgs)
   21. evaluate (local) fun(base::quote(`<smplErrr>`))
  Warning messages:
  1: replacing previous import 'data.table::first' by 'dplyr::first' when loading 'enchantr' 
  2: replacing previous import 'data.table::last' by 'dplyr::last' when loading 'enchantr' 
  3: replacing previous import 'data.table::between' by 'dplyr::between' when loading 'enchantr' 
  4: In normalizePath(file.path(report_params[["outdir"]], "assets",  :
    path[1]="assets/logo.png": No such file or directory
  5: In file.copy(report_params[["logo"]], target_logo, recursive = T,  :
    'recursive' will be ignored as 'to' is not a single existing directory
  6: In file.create(to[okay]) :
    cannot create file 'assets/logo.png', reason 'No such file or directory'
  
  Quitting from lines 221-243 [distToNearest_bulk] (_main.Rmd)
  Execution halted

Work dir:
  /rds/HvBBlpwx4dc/work/77/7340b6d640343813d5f5fa1479d69c

Container:
 /rds/hpc-work/dependencies/Singularity/docker.io-immcantation-airrflow-4.2.0.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details
ERROR ~ /tmp/nxf-18126076029403867543/sort.index (No such file or directory)

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: <https://nf-co.re/docs/usage/troubleshooting>

 -- Check '.nextflow.log' file for details
ERROR ~ /tmp/nxf-18126076029403867543/sort.index (No such file or directory)

 -- Check '.nextflow.log' file for details
ERROR ~ A DataflowVariable can only be assigned once. Use bind() to allow for equal values to be passed into already-bound variables.
The .nextflow.log contains this extra info:
Copy code
java.io.FileNotFoundException: /tmp/nxf-18126076029403867543/sort.index (No such file or directory)
	at java.base/java.io.RandomAccessFile.open0(Native Method)
	at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:344)
	at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:259)
	at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:213)
	at nextflow.file.SequentialFileStore.<init>(SequentialFileStore.groovy:53)
	at nextflow.file.SortFileCollector.makeHash(SortFileCollector.groovy:241)
	at nextflow.file.FileCollector.saveTo(FileCollector.groovy:239)
	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
	at nextflow.extension.CollectFileOp.emitItems(CollectFileOp.groovy:187)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
	at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
	at groovy.lang.MetaClassImpl.invokeMethodClosure(MetaClassImpl.java:1017)
	at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1207)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
	at groovy.lang.Closure.call(Closure.java:433)
	at groovy.lang.Closure.call(Closure.java:422)
	at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
	at nextflow.extension.DataflowHelper$3.afterStop(DataflowHelper.groovy:257)
	at groovyx.gpars.dataflow.operator.DataflowProcessor.fireAfterStop(DataflowProcessor.java:324)
	at groovyx.gpars.dataflow.operator.DataflowProcessorActor.afterStop(DataflowProcessorActor.java:59)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
	at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
	at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:633)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:624)
	at groovyx.gpars.actor.Actor.callDynamic(Actor.java:369)
	at groovyx.gpars.actor.Actor.handleTermination(Actor.java:320)
	at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:86)
	at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Feb-21 08:05:44.804 [Actor Thread 59] DEBUG nextflow.Session - Session aborted -- Cause: /tmp/nxf-18126076029403867543/sort.index (No such file or directory)
I'm just not sure if this is a failure due to the function or if its due to this missing /tmp/ file? Any suggestions on how to debug this further?
s
What I usually do to debug when the issue is not obvious, is I go to the working directory to check the input files look good, and they are not empty, then use the old school ways: run things at small increments and inspect. I open R in the container that is used for that process (so you need to launch singularity or docker, and be able to access the input files), and execute the .Rmd in a interactive session, so I can control all steps, and run code by line or by chunks, and inspect all intermediary variables. It seems the issue is happening in this chunk https://github.com/immcantation/enchantr/blob/master/inst/rstudio/templates/project/find_threshold_project_files/index.Rmd#L220.
g
Sorry for the late reply as I was away for some months, but when setting the manual threshold, you can also skip the 
REPORT_THRESHOLD
process if this one results in an error with the param 
--skip_report_threshold
.
Open in Slack
PreviousNext
Powered by