Hello, I would like to contribute to deepvariant/r...
# modules
n
Hello, I would like to contribute to deepvariant/rundeepvariant module. The output currently includes
.tbi
index, google/deepvariant also supports
.csi
index when the size of any reference chromosomes exceed 512M: https://github.com/google/deepvariant/blob/r1.9/deepvariant/postprocess_variants.py#L1387-L1415 The changes I intend to push are described in issue#8694. Shall I continue? I was thinking that many people use this module, and the change could cause linting/channel-calling problems to their local repository.
👍 1
r
sounds good
You can make the output channel optional. People who want to use it can then access it, but it won't break downstream code
👍 1
n
Good idea! Instead of replacing all the
tbi
, I can add
index
in
meta.yml
and
main.nf
as an optional output. Thank you Rike! I'll update here when I create a PR.
s
Shall I continue? I was thinking that many people use this module, and the change could cause linting/channel-calling problems to their local repository. (edited)
When people update the modules in their pipeline they should check to ensure that any changes to the module have don't break their downstream. Definitely update and improve the module, using
index
instead of
tbi
seems perfect.
👀 1
m
I'm wondering what do we do in other modules that output csi or tbi
s
Use index as a general rule is better
party this 2
IMO
m
I'm happy too, just want to be coherent
s
I had a discussion about this when I was porting the sentieon modules, not sure if I can find it again
a
Other modules (that I've used) often seem to have both
tbi
and
csi
channels with each being optional. My personal opinion is that separate channels make understanding what is in them much clearer, allow for cases where the different channels might need to be handled differently without any complex parsing of
args
or file extensions. So for example maybe I have
.index
and I need to do something different depending on whether I get
csi
or
tbi
- I would need to write some boilerplate like
Copy code
DEEPVARIANT_RUNDEEPVARIANT.out.index
   .branch{ meta, idx ->
    csi: idx.name.tokenize('.')[-1] == 'csi'
    tbi: idx.name.tokenize('.')[-1] == 'tbi'
}
The spec prefers single-channel but I respectfully think the spec is wrong here and that is my #small-mound-i-feel-strongly-about-but-not-enough-to-fight-everyone
s
Can you give an example of when that actually happens @Arthur? Personally I find it the opposite, that you have to manually join together what are essentially the same channel (I also find it extremely annoying that we have files and their index emitted separately)
a
I agree on the latter point and I'd probably be inclined to accept a single channel I had to branch in that situation
s
Yes, and my personal #small-mound-i-feel-strongly-about-but-not-enough-to-fight-everyone is that is wrong
m
Ok, so that sounds like a discussion for our next maintainers meeting
a
Just fight each other? 😂 As usual I will ultimately acquiesce to the spec but would prefer if there was consistency and I think on this issue there may be quite a strong split
👍 1
m
Consistency is my #hill-to-die-on
But we need to agree on specs
a
I think maybe the problem here is spec says
SHOULD
and without someone militantly converting existing cases where no good reason to diverge exists, people just go with their preference. Also I think samtools/index and tabix/tabix both are separate and these are indexing modules people probably copy when unsure? (I know that's what I've done)
💯 2
n
DeepVariant itself emits
.tbi
if
max_chromosome_length < 5e8
, emits .
csi
if
max_chromosome_length > 5e8
. Our case is that
rundeepvariant
moduleI currently limits the emit by naming it to be
.tbi
only, therefore we need to add a
.csi
. It sounds like either use
index
universally; or name every format of index output, which is what
Samtools/index
module focuses on (creating different types of index) so it takes that effort.