OK lots of pipeline has the same issue
# tools
j
OK lots of pipeline has the same issue
m
when you say lots?
j
34/100 failed
m
these are all because they are too outdated (if I remember correctly, the main culprit is a missing `.nf-core.yml`file), once this is there, they are picked up again
j
createtaxdb defintiely has a recent .nf-core.yml file
(and I resynced 10 minutes ago to check)
m
ah seems something is outdated with the nextflow config https://github.com/nf-core/tools/actions/runs/12431872066/job/35357930427#step:8:45
j
stuff like funcscan/eager may not have the latest thing
what does that mean? ๐Ÿ˜…
p
Looks like Nextflow v24.11.0-edge can't parse the config file
m
letโ€™s wait for @Jรบlia Mir, she knows best about sync issues
๐Ÿ‘ 1
p
So the pipeline is likely broken currently
j
We've bumped up to an edge version?
34 pipelines are brokne ๐Ÿ˜…
Ah yeha latest-everything
๐Ÿ‘๐Ÿป 1
ignore me
p
We always use edge versions for CI.
Command being run is this:
Copy code
nextflow config -flat path/to/createtaxdb
It just dumps the parsed Nextflow config
j
INteresting, works on
dev
So must be something specific to the template branch
j
having a look now, the missing
.nf-core.yml
is making some of these pipelines fail, it is known and we posted a message to these pipelines channels. But the createtaxdb is something different
j
eager, funcscan all should have
.nf-core.yml
I think too
j
eager and funcscan failed due to a pre-commit error, and other pipelines too: โ€ข mag โ€ข pathogensurveillance โ€ข pixelator โ€ข reportho โ€ข taxprofiler โ€ข troughgraph โ€ข fastqrepair โ€ข denovotranscript โ€ข eager โ€ข funcscan but I opened the PR manually to them and should fix this error for the next version
maybe I forgot createtaxdb? ๐Ÿค” let me investigate ๐Ÿ”
j
Ooooh ok (embarrassing that covers all my pipelines ๐Ÿ˜… )
๐Ÿ˜ 4
What was I doing wrong?
j
not your fault, I think it was the template ๐Ÿ™ˆ
it's affecting all (or almost all?) the pipelines that skip igenomes
m
#references skips igenomes, but it wasn't a pipeline until yesterday
but I did manage to merge a manual template sync yesterday
j
Ahhhhh
stupid igenoems
pain in my butt
m
that being said, @Jรบlia Mir I noticed an issue
I have igenomes in skip_features, but I noticed a warning for conf/igenomes.config and conf/igenomes_ignored.config with nf-core lint
๐Ÿ˜ฑ 1
let me reproduce
Copy code
โ•ญโ”€ [!] 11 Pipeline Test Warnings โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚                                                                                                                                                                                                                                         โ”‚
โ”‚ files_exist: File not found: conf/igenomes.config                                                                                                                                                                                       โ”‚
โ”‚ files_exist: File not found: conf/igenomes_ignored.config                                                                                                                                                                               โ”‚
with:
Copy code
skip_features:
    - igenomes
you want an issue?
f
You need :
Copy code
lint:
  files_exist:
    - conf/igenomes.config
    - conf/igenomes_ignored.config
to ignore the linter warning
m
that's what I did actually, but there should not be a warning
there wasn't one in 3.0.2
a
That
lint
section is present in the current release template features yml. So why you get a warning is unclear?
m
I shouldn't get a warning for igenomes related files if I skip this feature
that's what I meant
a
m
that's because I used the files_unchanged
a
I assume you mean
files_exists
, but my point is that is part of the template if you skip
igenomes
?
๐Ÿ‘ 1
m
just adding things to skip-features, doesnโ€™t exclude them from linting (this is because the whole logic is mostly for
nf-core pipeilnes create
and not so much for manual edditing), so you need to manually add all necessary execptions
a
Ummโ€ฆ actually I think it is supposed to and thatโ€™s my point!See this line
Iโ€™ve tested this locally now though and evidently that method is not working in
sync
. I am unclear as to whether it isnโ€™t running or whether the config is being overwritten with the old content somehow
Ok I think I found the issue. New GH Issue.
๐Ÿ‘€ 1
j
thanks for digging into it @Arthur
j
OK wait... I'm lost now... do I need to do anything for my issue?
j
not yet ๐Ÿ™‚ I have lots of meeting today so I will investigate later/tomorrow, but there is an issue about it already and we will fix it for the next patch release
j
OK! no worries and thank you ๐Ÿ˜„