Hi! I'm currently updating the chopper module. In ...
# modules
o
Hi! I'm currently updating the chopper module. In the [documentation](github.com/wdecoster/chopper#…), it's advised to use pigz as a replacement for zcat + gzip. Since pigz is used twice in the module (once for decompressing, a second time for compressing output):
Copy code
pigz \\
        -dc \\
        -p ${task.cpus} \\
        $args \\
        $fastq | \\
    chopper \\
        --threads $task.cpus \\
        $fasta_filtering \\
        $args2 | \\
    pigz \\
        -p ${task.cpus} \\
        $args3 > ${prefix}.fastq.gz
I need to define it twice in the meta yaml:
Copy code
tools:
  - "pigz":
      description: "A parallel implementation of gzip for modern multi-processor, multi-core machines."
      documentation: "<https://zlib.net/pigz/>"
      args_id: "$args"
      identifier: ""
  - "chopper":
      description: "A rust command line for filtering and trimming long reads."
      homepage: "<https://github.com/wdecoster/chopper>"
      documentation: "<https://github.com/wdecoster/chopper>"
      tool_dev_url: "<https://github.com/wdecoster/chopper>"
      doi: "10.1093/bioinformatics/bty149"
      licence: ["MIT"]
      args_id: "$args2"
      identifier: ""
  - "pigz":
      description: "A parallel implementation of gzip for modern multi-processor, multi-core machines."
      documentation: "<https://zlib.net/pigz/>"
      args_id: "$args3"
      identifier: ""
The thing is, once the replacement of both zcat and gzip by pigz is done, I get an error with the linter:
Copy code
main_nf_version_emit: Module does not have an emit: and topic: for each software version
In your opinion, how should I proceed? Thanks!
n
I think
pigz
is only parallelising compression and does not offer a meaningful benefit during decompression. So only using it for compression might be one option
k
I think it might be easier to look at your code changes to help you here. Would you mind sharing a draft PR? Multiple usage of the same tool shouldn't be a problem in the same process. To me it seems rather that you don't emit the software version via topics?
n
I also think that the information in meta.yml as it currently is is not very helpful, because it doesnt make clear whats different betweena args and args3, is
args_id = ["args", "args3"]
allowed?
Another option would be of course to let handle
chopper
decompression of the input, appearently it can just deal with a gzipped input via
-i
?
o
Yes pigz is also way faster that gzip for decompressing. As stated in the [chopper documentation](github.com/wdecoster/chopper#…), it's better to use pigz both sides. Thanks @Niklas Schandry for the tip! Writing
Copy code
args_id:
    - "args"
    - "args3"
did the trick!