Hi, I have the following PR where the following pa...
# team-maintainers
l
Hi, I have the following PR where the following parameter is defined in a python script.
opt["dataset"] = "${task.ext.dataset ?: meta.dataset}"
The issue I see with such pattern is that the user needs to pass the value either by the ext arguments or by the meta map. I have recommended to use a value channel instead to allow a description of the input in the yml file. This is similar to @Mahesh Binzer-Panchal issue with ext arguments, what are the different key of the meta map that should be allowed ? What should I answer to the contributor ?
The dataset is always expected to be provided, and it has already been included in the meta.yml via the
meta
map.
https://github.com/nf-core/modules/pull/8698
s
what is dataset, a string?
l
It seems so !
Copy code
dataset_id = opt["dataset"]
processed = input_processing(data, dataset_id, genome)

## main.nf.test

input[0] = [
                    [ id:'test', dataset: 'HCC1395' ], 
                    file(params.modules_testdata_base_path + 'delete_me/sigprofiler/test_GRCh38/HCC1395_WXS_HCC1395_joint_table.tsv', checkIfExists: true),
                ]
s
is it just using it to name files in some way?
or is it being used inside the code in some other way?
A quick look suggests it is being used to set columns and to find the input file? (maybe) I suspect this could be
prefix
l
It seems to be quite used in the script to set as a value in a table,
Copy code
new_columns = {'Project': dataset_id, 'Genome': genome, 'Type': "SOMATIC", 'mut_type': "SNP"}
    df = data.assign(**new_columns)
It's also use as input to compute some kind of matrix
Copy code
generate_count_matrix = (
        f"SigProfilerMatrixGenerator matrix_generator "
        f"{dataset_id} {genome} {input_path} --volume {opt["volume"]}"
    )
I guess that it could also be prefix as prefix is only used to create a folder.
f
I think it should be on an input channel. It seems to be required to run which points me to a channel. I would also remove it from the meta map as
meta.dataset
it is not a standard field as far as I know. Not sure about
prefix
. The parameter seems to be used for things other than naming output files.
i
Custom meta fields are not allowed for a good reason, and the
task.ext
feels hacky. If it's a mandatory input, then I see no reason not to do it by the book and make it a channel.
👍 1
this 2
l
I totally aggree, it just that the contributor doesn't seems to see it this way 😄
i
I've added a longer explanation of the rules, let's see their response
m
Also a reminder that
task.ext
maps are intended for users to override pipeline developer settings.