Do we not have a guideline for the use of `task.ex...
# team-maintainers
m
Do we not have a guideline for the use of
task.ext.*
? I see there's several modules with fields that aren't part of what I thought was the agreed list of fields.
Copy code
% find modules -name "*.nf" -exec grep -H "task.ext." {} \; | grep -vE "task\.ext\.(singularity|args|prefix|when|suffix)"
modules/nf-core/picard/fixmateinformation/main.nf:    def STRINGENCY = task.ext.stringency ?: "STRICT"
modules/nf-core/artic/minion/main.nf:    def hd5_plugin_path = task.ext.hd5_plugin_path ? "export HDF5_PLUGIN_PATH=${task.ext.hd5_plugin_path}" : "export HDF5_PLUGIN_PATH=/usr/local/lib/python3.6/site-packages/ont_fast5_api/vbz_plugin"
modules/nf-core/fgbio/zipperbams/main.nf:    def compression = task.ext.compression ?: '0'
modules/nf-core/jupyternotebook/main.nf:    def parametrize = (task.ext.parametrize == null) ?  true : task.ext.parametrize
modules/nf-core/jupyternotebook/main.nf:    def implicit_params = (task.ext.implicit_params == null) ? true : task.ext.implicit_params
modules/nf-core/jupyternotebook/main.nf:    def meta_params = (task.ext.meta_params == null) ? true : task.ext.meta_params
modules/nf-core/jupyternotebook/main.nf:    def kernel   = task.ext.kernel ?: '-'
modules/nf-core/rmarkdownnotebook/main.nf:    def parametrize = (task.ext.parametrize == null) ?  true : task.ext.parametrize
modules/nf-core/rmarkdownnotebook/main.nf:    def implicit_params = (task.ext.implicit_params == null) ? true : task.ext.implicit_params
modules/nf-core/rmarkdownnotebook/main.nf:    def meta_params = (task.ext.meta_params == null) ? true : task.ext.meta_params
modules/nf-core/plink2/vcf2bgen/main.nf:    def reffirst = bgen_reffirst || task.ext.reffirst ? "ref-first" : ""
modules/nf-core/plink2/vcf2bgen/main.nf:    def dosage = task.ext.dosage_field ? task.ext.dosage_field : dosage_field
modules/nf-core/plink2/vcf2bgen/main.nf:    def sample_name_opt = task.ext.sample_name_mode ? task.ext.sample_name_mode : sample_name_mode
modules/nf-core/hmmer/eslreformat/main.nf:    def postproc = task.ext.postprocessing ?: ""
modules/nf-core/cellbender/merge/main.nf:    output_layer = task.ext.output_layer ?: "cellbender"
modules/nf-core/cellbender/removebackground/main.nf:    container "${ task.ext.use_gpu ? '<http://us.gcr.io/broad-dsde-methods/cellbender:0.3.2|us.gcr.io/broad-dsde-methods/cellbender:0.3.2>' :
modules/nf-core/cellbender/removebackground/main.nf:    use_gpu = task.ext.use_gpu ? "--cuda" : ""
modules/nf-core/bedtools/groupby/main.nf:    def summary_col = task.ext.summary_col ? "-c ${task.ext.summary_col}" : "-c 5"
modules/nf-core/scvitools/scar/main.nf:    input_layer = task.ext.input_layer ?: "X"
modules/nf-core/scvitools/scar/main.nf:    output_layer = task.ext.output_layer ?: "scar"
modules/nf-core/scvitools/scar/main.nf:    max_epochs = task.ext.max_epochs ?: ""
modules/nf-core/scvitools/solo/main.nf:    container "${ task.ext.use_gpu ? '<http://docker.io/nicotru/scvitools-gpu:cuda-12|docker.io/nicotru/scvitools-gpu:cuda-12>' :
modules/nf-core/scvitools/solo/main.nf:    batch_key = task.ext.batch_key ?: ""
modules/nf-core/scvitools/solo/main.nf:    max_epochs = task.ext.max_epochs ?: ""
modules/nf-core/metaspace/converter/main.nf:    database_name    = task.ext.database_name ?: "HMDB"
modules/nf-core/metaspace/converter/main.nf:    database_version = task.ext.database_version ?: "v4"
modules/nf-core/metaspace/converter/main.nf:    fdr              = task.ext.fdr ?: "0.1"
modules/nf-core/metaspace/converter/main.nf:    use_tic          = task.ext.use_tic ?: "true"
modules/nf-core/metaspace/converter/main.nf:    metadata_as_obs  = task.ext.metadata_as_obs ?: "false"
modules/nf-core/sentieon/dedup/main.nf:    def metrics = task.ext.metrics ?: "${prefix}.metrics"
modules/nf-core/sentieon/dedup/main.nf:    def metrics = task.ext.metrics ?: "${prefix}.metrics"
modules/nf-core/icountmini/segment/main.nf:    def regions_prefix = task.ext.regions_prefix ?: "${gtf.simpleName}"
modules/nf-core/icountmini/segment/main.nf:    def regions_prefix = task.ext.regions_prefix ?: "${gtf.simpleName}"
👀 2
shakefist 3
l
It seems to often be an issue of an optional parameter that isn't natively present in the tool. The issue I see with this, is that it isn't documented anywhere. I would prefer to force them to be as a value channel and document them in the meta.yml.
💯 3
f
maybe open an issue to track these?
f
This comes back to the eternal confusion of users with
ext.args
I believe and the fact that we could probably document it better. @Mahesh Binzer-Panchal What would be your preferred way to handle these? Slap on the wrist and open issues to make them ext.args? I had a case yesterday where a module was implemented as local instead of using the nf-core module and had some optional parameters completely hardcoded in the script block. So there are other variations of this issue for sure around 🤷
n
About `task.ext.use_gpu`: This is actually part of the pipeline template
m
Interesting. When did that get added?
What would be your preferred way to handle these? Slap on the wrist and open issues to make them ext.args?
I think this depends on the context. For the majority, I feel like they should be inputs
n
As part of #CE5LG7WMB 3.3.0 with this PR
m
Should the user be able to override
task.ext.use_gpu
? i.e. remove the
--cuda
option?
The original intension of using
task.ext
was to allow a simple way for users to override module settings set by the developers.
n
The basic idea is that
process_gpu
signalises that a process supports running on a GPU. The
task.ext.use_gpu
is then used to signalize to the process that it actually should use it. This will generally be the case if the
gpu
profile is enabled. Thus, the GPU profile is "forwarded" via
task.ext.use_gpu
. However, in some cases one might want to use the GPU in general, but for some processes the GPU does not have enough VRAM or something. Then the user can override
task.ext.use_gpu
for single processes.
👍🏽 1
m