Mirroring a discussion that we had at a PR for a new module:

There seems to be a tradeoff between enforcing the

prefix

appended to the start of outputs and losing the ability to specify the file extension, e.g.

--output-cell-metadata ${prefix}cell-metadata.csv.gz

An alternative is to allow the process to take an

output format

as an input variable, then specify the formats for all outputs using that, but that loses out on the ability to granularly control each file individually. Is it sufficient to just not enforce the file-extension prefixes for this process? We could leave it as the default for the one file that needs to be manually specified to avoid collisions with output glob detection, then allow the user to specify it for other files with task.ext if they wanted

Any thoughts on this? The PR is: https://github.com/nf-core/modules/pull/8762 (would be helpful to know what others think so that this module works for most approaches).

I had a brief look at the PR and comments, but I can't quite distill what the two options are that being discussed here

When I understand it correctly either the user sets these flags from outside and then they can control basically the whole filename + extension or the user just sets a prefix and then the file extension has to be set beforehand. The question is on what level of flexibility the module needs to offer (I think)

Yes there are flags for the output file(s) name and a separate flag for format. If a file name is provided, the file will be of that type, but the extension will not be enforced appropriately: it will just use the name provided. If I want to name the ouptut files

${prefix}-output.csv.gz

but the user wants the output in

.parquet

format, the name will not reflect that. There are ~5 files that have this name + extension option combo. The options I see: 1. Don't add the

$prefix

to each file by default. This gives a user full control of naming and extension a. There is 1 file that must be renamed to be uniquely captured by the output channel globs, but it does not have format options. So whether or not this file uses the prefix is a separate question (if no other outputs will), but it does not relate to the other question of format/name collision 2. Do specify the

$prefix

for each file, locking in the file extension name for all files. We could also optionally: a. Allow the user to specify a global output format b. Have each file's output format be an input Basically, is it worth adding a bunch of input vars (specifying each output file type) to make the full suite of options available if the

${prefix}

must be appended to each output file? There's some tradeoff between simplicity, freedom, and compliance. Personally, I'm not sure of the use case of outputing the files in different formats, so I'm of the opinion that 2a is sufficient. If we aren't worried about enforcing

$prefix

, then we can just not worry about this whole thing, but I'm not sure how strong of a requirement that is for nf-core compliance

So is the format for each file controlled by a second per-file option? Eg (

--stats-file-format

) If so it would be somewhat trivial imo to have an input channel that takes a tuple of vals representing each output file type? This could then be used to determine the correct extensions and the correct flags

Yes most files have a second, per-file format specification. So it sounds like the preferred option is to have an array of file extensions passed as input, then manually specify all of the output files name & format? option 2b?

I would personally just use the extensions tuple to determine whether optional ones are output - if no extension is supplied for a file, the arg for it isn't set. But this is just my thoughts, maintainers may have a different take here

would that be an 'optional input' then? Or something like if the string is empty, don't output the file

I was imagining something like:

input:
...
tuple val(stats_fmt), val(seq_fmt), val(report_fmt)

output:
...

script:

def stats_args = stats_fmt ? "--stats-format ${stats_fmt} --stats-file ${prefix}.somename.${stats_fmt}" : ""

"""
tool \\
    --some-arg
    ${stats_args} \\
    ...
"""

See the PR now, passing tests with the new input channel specification. My take is that the optional files are unlikely to be used, and requiring their specification may overcomplicate the module. Adding an input specification for optional/mostly blank fields seems clunky and the optional files can still be specified using task.args if needed. But perhaps the preference is for explicitness over parsimony. If that is the case, I'll make the change.