Hi I encountered the following issue when linting ...
# tools
n
Hi I encountered the following issue when linting with the latest dev branch:
LookupError: Could not find branch information for component 'happy/happy/happy'.Please remove the 'modules.json' and rerun the command to recreate it
I already recreated the modules.json file but the error persists.
a
What lint command are you running, and which tools version?
n
nf-core pipelines lint
on the current dev branch ๐Ÿ™‚
a
Hmm, any idea which of the lint tests is actually throwing the error?
n
This is the stacktrace
Untitled
(don't know what went wrong with the first snippet ๐Ÿ˜›)
so
module_changes
apparently
Seems like it only fails for certain modules which were initially installed via subworkflows. I managed to remove the
happy/happy
module because it wasn't used anywhere anymore but now I have the same error with
snpeff/snpeff
(also a modules I do not use in my pipeline but got installed via a subworkflow)
a
Same error?
snpeff/snpeff/snpeff
has no branch info?
n
yes
LookupError: Could not find branch information for component 'snpeff/snpeff/snpeff'.Please remove the 'modules.json' and rerun the command to recreate it
I also removed and reinstalled the affected subwf, but that didn't do anything
a
Ok so this is some weirdness where
<http://module.org|module.org>
must be being set to e.g.
snpeff
n
Yeah seems like such a thing, weird bug
a
And maybe itโ€™s just coincidence, but seems odd that it happens so far with modules where
tool==subtool
n
yeah also true ๐Ÿค”
I'll see if I can find the exact issue (wanted to post about it first before doing that)
a
Found the issue. It is indeed due to
tool==subtool
. https://github.com/nf-core/tools/blame/25224a03db8d16767e4241982f6bf66829a03b21/nf_core/components/nfcore_component.py#L67-L72 Looks like the guilty code went in 5 days ago so makes sense you see the error now
n
Nice that was fast!
j
@Nicolas Vannieuwkerke you missed a dad joke! Tools isn't happy because it can't find happy/happy/happy!
๐Ÿคฃ 1
n
Love how you casually join the conversation to shame me into missing a dad joke ๐Ÿ˜‰ (while casually dropping one yourself ๐Ÿคฃ)
๐Ÿ’ฏ 1
@Arthur are you fixing this or should I do it? ๐Ÿ˜›
j
I'm running on very little sleep and trying to stop my son from killing himself, there is not much to live for at the moment
n
That explains the urge to drop some dad jokes ๐Ÿ˜‰
keep 'em coming!
a
The simplest fix (I think) is to apply a slightly different index search:
Copy code
from operator import indexOf

def rindex(li, value):
    return len(li) - 1 - indexOf(reversed(li), value)
And then:
Copy code
name_index = (
                len(self.component_dir.parts)
                - 1
                - rindex(self.component_dir.parts[::-1],self.component_name.split("/")[0])
            )
SCRATCH THAT: The current code is actually already reversing the list to find the index of the โ€œlastโ€ occurrence of the component name. So actually we can just find the โ€œfirstโ€ using:
Copy code
name_index = self.component_dir.parts.index(self.component_name.split("/")[0])
Caveat that I havenโ€™t looked into why the changes were implemented in the first place so maybe something more needs to be done. @Jรบlia Mir may know.
๐Ÿ‘ 1
n
do you have a code location for that? I'll try it out on my pipeline and see what it does
n
thanks
a
I think the issue is that Julia implemented this to handle the case where the
org==tool
maybe? And then my solution above also fails for the reverse reason
Yup indeed it appears this is the issue: https://github.com/nf-core/tools/pull/3147 (Slack thread)
๐Ÿ‘ 1
n
I'm currently looking into a possible fix that resolves both issues
a
Best thing I came up with off the top of my head is to split the directory as a string on the
component_name
then take the last element of it split by
/
. But itโ€™s not very clean.
n
I already tried that but that isn't the issue (since in the bug's case the first and last element are the same)
This does seem to work:
Copy code
component_list = self.component_name.split("/")

            name_index = (
                len(self.component_dir.parts)
                - 1
                - self.component_dir.parts[::-1].index(component_list[0])
            )
            if len(component_list) != 1 and component_list[0] == component_list[1]:
                # Handle cases where the subtool has the same name as the tool
                name_index -= 1

            repo_dir = self.component_dir.parts[:name_index][-1]
It checks if the tool and subtool are the same and in that case it moves the index one back so the org is actually correctly parsed
And the linting finishes succesfully with that fix
(linting errors though but that's my problem ๐Ÿ˜)
a
Ok I have something which should work:
Copy code
tool, subtool = self.component_name.partition("/")
    if subtool and (tool == subtool):
        idx = 1
    else:
        idx = 0
    
    name_idx = (
        len(self.component_dir.parts)
        - 1
        - self.component_dir.parts[::-1].index(tool, idx)
    )
(
.index
can take a starting or end idx to limit the search space to whilst still returning the actual index)
n
https://github.com/nf-core/tools/pull/3156 => with a very similar solution to yours
j
Reading this now! but I see you already found why I introduced the change ๐Ÿ•ต๐Ÿปโ€โ™‚๏ธ I'm not on my PC, but both of your solutions seem good, thanks for fixin it!
๐ŸŽ‰ 1
n
(it's failing for a changelog but nothing needs to be added to the changelog here ๐Ÿ˜›)
Thanks @Jรบlia Mir!