https://nf-co.re logo
Join Slack
Powered by
Hi Huimin <@U08KDH7QJSV>, thank you, it seems to m...
# airrflow
r
Hi Huimin @Huimin Lyu, thank you, it seems to me that FIND_CLONAL_THRESHOLD is serial. We do have a single CPU with a lot of memory. I gave it more CPUs but the process does not seem to be using them, and I don't see any configuration about that online
h
The link you gave me is the website for shazam, the tool we used in nf-core/airrflow to find threshold. Check our nf-core/airrflow (https://nf-co.re/airrflow/) website for all workflow documentations. The configuration is not within the documentation through. For your case, no matter how large amount of memory you request, with 1cpu, only one subject is able to be processed at a time taking the memory it needs for single processing.
r
Thanks Huimin. I think I understand now where the disagreement is. Looking at the source: 
<http://clonal_analysis.nf|clonal_analysis.nf>
→ 
<http://find_threshold.nf|find_threshold.nf>
→ 
enchantr::findThreshold()
→ 
shazam::findThreshold()
— I don't see any internal parallelization within the threshold computation itself, which would explain why adding CPUs didn't change 
AveCPU
in 
sstat
. You may be right that multiple CPUs allow parallel subject-level processing (separate R processes per subject), but in our case the job tag is 
all_reps
, which I think means that all 175 samples are being processed together as a single task rather than split by subject. If that's the case, extra CPUs can't help because there's only one task to run. Does airrflow have a way to split the threshold computation by subject or group rather than running 
all_reps
as a single job? That seems like it would be the real fix for a dataset this size, rather than tuning resources.
h
Could you please share with me your samplesheet and the command you used to run nf-core/airrflow?
Actually, it occurred to me that although the workflow reports threshold for each subject by default, distances within subject and across subjects are calculated to determine the threshold for each subject. That's why this step show 
all_reps
. I need to check the papalism in this step. A temporal solution is that you provide a threshold (we use 0.1 a lot) and skip threshold report.
--clonal_threshold 0.1
--skip_report_threshold true
r
Ok, will run it with those flags to get some results. And maybe after that I will try to run it for a week and see if it finishes.
This is what I am using now:
Copy code
nextflow run nf-core/airrflow -r 5.0.0 \
  -profile nebnext_umi_bcr,apptainer \
  --input airrflow_samplesheet_bcr.tsv \
  --outdir /users/r/b/rbarrant/projects/provideintendtcrbcr/airr_results_bcr/ \
  --igenomes_ignore true \
  -w /users/r/b/rbarrant/scratch/airrflow-work \
  --clonal_threshold 0.1 \
  --skip_report_threshold true \
  -resume
the samplesheet is pretty long as it has 175 samples
h
With 175 samples, the threshold calculation will take a long time. We are implementing faster algorithm to do pairwise distance calculation. We will have a new nf-core/airrflow release soon. The new pairwise distance calculation algorithm will be there to speed up clonal assignment. However, it will not be there for threshold calculation. We will test and update Shazam package first and implement it in the future nf-core/airrflow release.
BTW, there's no igenomes_ignore parameter in nf-core/airrflow. You can check the available parameters here: https://nf-co.re/airrflow/5.0.0/parameters/
r
Thank you!! This is great!
2 Views
Open in Slack
PreviousNext
Powered by