How to handle a list of values in an input channel...
# modules
c
How to handle a list of values in an input channel? In one module that I develop (
last/mafconvert
), the module takes a file format as input and converts another input to that format. I am planning to expand the module so that it takes a list of formats, because my users sometimes want multiple formats for the output. For instance, if it gets a file
foo.maf
and the formats gff, sam and axt as input, it will create a file
foo.gff
, a file
foo.axt
and file
foo.sam
as output. I want the list of format to be able to take any number of formats from a fixed list of possible values. My questions are how to handle that in the module and how to handle that in a pipeline using the module? • In the module, is it better to take a character-delimited string such as
sam
,
axt,sam
or
sam,axt,gff
, etc? Or to take an array of values (
['sam']
,
['axt', 'sam']
,
['sam', 'axt', 'gff']
, …) • In a pipeline, ◦ is there already a standard for passing a list of values through a parameter? ChatGPT suggests me character-delimited strings and then
.split(',')
or
.tokenize(' ')
. \ ◦ And What would be the best way to handle this in the
nextflow_schema.json
?
a
Can't speak to how it might currently be done in any modules, but I think a few factors are relevant here. • how many possible formats are there? (Would it be feasible to pass each as it's own channel for example) • Is there any restriction on compatibility of formats used at the same time? • what does the expected use of the module look like in a real pipeline? For the schema part - the schema can handle arrays but currently the nextflow CLI cannot (afaik) take multiple params. This leaves you with a couple of options: • use an array in the schema and people will have to use
params.json
(or yml) to supply a real array (they could still set a single value on the cli I think) • Use a comma-separated string and
.split
and do validation of the actual values in the pipeline
👍 1
c
Thanks for the feedback! The permitted values are "axt", "bed", "blast", "blasttab", "blasttab+", "chain", "gff", "html", "psl", "sam", "tab", plus soon "bam" and "cram". Some users of the #pairgenomealign pipeline want more than one of them, because they use multiple downstream tools that are narrow and unique in what they accept as input. For instance one may want Axt format for the CNEr package, Chain for Jbrowse and CRAM as an alternative to MAF (which takes too much disk space). I will try a comma-separated parameter and propose a PR where the module takes an array of values in the input channel that is currently taking a single value.