I'm trying to migrate the `fasta_index_dna` subwo...
# rubber-duck
s
I'm trying to migrate the
fasta_index_dna
subworkflow to nf-test. This is proving trickier that I'd expect. @Matthias De Smet, you originally implemented this module by the looks of it. This subworkflow takes the fasta, but also an extra altliftover channel:
Copy code
ch_fasta        // channel: [mandatory] [ val(meta), path(fasta) ]
        ch_altliftover  // channel: [mandatory, if aligner is bwamem or bwamem2 or snap] [ val(meta), path(altliftover) ]
The issue is in this section of the code:
Copy code
BWAMEM1_INDEX.out.index
                .join(ch_altliftover)
                .map { meta, index, alt -> [meta, index + alt] }
which I believe is trying to add the altliftover file inside the folder that is output by bwamem. But I don't think we can use
+
to do this, because it is a file not a meta map, and what we get from nf-test begins (condensing and truncating for space):
Copy code
"0": [[{"id": "test"},
                        ["/workspace/modules/.nf-test/tests/fcebacc24247d7e952390921d1be834a/workspace",
                         "/workspace/modules/.nf-test/tests/fcebacc24247d7e952390921d1be834a/modules",
                         "/workspace/modules/.nf-test/tests/fcebacc24247d7e952390921d1be834a/.nf-test",
                         "/workspace/modules/.nf-test/tests/fcebacc24247d7e952390921d1be834a/tests",
                         "/workspace/modules/.nf-test/tests/fcebacc24247d7e952390921d1be834a/fcebacc24247d7e952390921d1be834a",
                            [[[
                               ".command.begin:md5,d41d8cd98f00b204e9800998ecf8427e",
i.e. it returns a list of all the files and folders with full paths. If I actually give a file to ch_altliftover, it still gives full paths but only the new file not the original bwa folder at all. I don't know how to pass this file inside the folder, without adding an extra process.
m
Sorry, can you clarify a bit more what the intended output is from given inputs? I'm not quite understanding. If you're trying to add a file to folder, then keep in mind that if the folder is passed as a input to process, adding files to it will break reentrancy.
s
I think the idea is that the altliftover file should be being put inside the bwa/ folder, alongside the files that are generally returned by bwa
m
Hi Simon, The point is to add the altliftover file to the output of the
*_INDEX
processes (in the case of
BWAMEM
and
BWAMEM2
) and to add it as input to the index process for
SNAPALIGNER
. All three of those support lifting alignments for alt contigs to the canonical chromosome, if the altliftover file is available
s
Yes, but I don't think the current code is functional as is
m
Can the liftover file not be an extra channel?
m
I think it has to sit in the same folder as the index, no?
šŸ‘ 1
I feel that's a similar issue as the fasta file for the bismark index
m
I was just going to say that
m
shared a GIF
s
Basically something needs to change here I think. One of: • Add an extra process in the subworkflow to add the alt file into the bwa folder • Add an extra input to bwa index, to take the alt file and just pass it through into the bwa output folder • Add an extra input to bwa mem to use the alt file as a separate channel or element of the index channel
Now I think the least disruptive option is to add an extra process into the subworkflow
m
from the help of `bwamem2`:
Copy code
-j            treat ALT contigs as part of the primary assembly (i.e. ignore <idxbase>.alt file)
Which led me to believe the the
*.alt
file needs to be in the same directory and use the same naming scheme as the rest of the index
šŸ’Æ 1
party this 1
m
m
As far as I know, we have the alt in the bwa folders for GATK/GRCh38
s
So, what do we do with this subworkflow?
m
ok, so for me it's more a question of where do we want the alt to sit
once we agree on that side, we deal with the subworkflow
s
It needs to go into the bwa/ folder
m
so for me it's more: • alt with build indexes • alt separated from build indexes Once we figure how to properly deal with this, we can adapt the subworkflow
šŸ‘ 1
my main issue with having it in the build indexes folder by default is that it's totally hidden
s
igenomes has it in the index folder for GATK/GRCh38
m
I know, I put it there
But how do you know just by looking at the references?
s
You don't
m
which I don't like
s
Could it be made inside the bwa/index process?
m
we can always figure a way to make it inside
but that sounds like a good idea
if a alt file is supplied we put it in the bwa output folder
but should it be at the index level or at the align level?
I am in favor of doing this at the align level, and adding this file as an extra optional reference
that sounds like a #C073ZDLNC92 thing
s
yeah, we could do either way, we just need to make a decision
m
that sounds perfect for a discussion on the maintainers meeting on Friday
but you're off right?
(By off, I mean un-available for nf-core)
s
Yeah, I'm at a board game convention
šŸ™ŒšŸ½ 1
šŸ™Œ 1