Hi ! I'm fixing the linting of the phaseimpute pip...
# tools
l
Hi ! I'm fixing the linting of the phaseimpute pipeline with the new version of nf-core tools. There is multiple warnings that I didn't had before most of them are legit but some of them are a bit strange. Here is a few example: • Included component
completionEmail
,
completionSummary
, ... not used in main.nf (for utils_nfcore_phaseimpute_pipeline) but they are used in the
PIPELINE_COMPLETION
workflow. Does tools checks only for the first workflow present ? • Included component
UTILS_NEXTFLOW_PIPELINE
,
UTILS_NFCORE_PIPELINE
and `UTILS_NFSCHEMA_PLUGIN`versions are not added in main.nf but they don't emit any. Should they be change to a function therefore ? • I have a few: Subworkflow includes less than two modules. But these subworkflow does heavy channel reconfiguration / merging. Should I set the channel modification as a function ? and where should I put these functions ?
a
Do you have a link to these failing lints in CI?
a
This is a bit weird because when I look at the 3.2.0 template merge for e.g. RNASeq I don’t see that
pipelines lint
runs the module/subwf linting 🤔
l
That's also strange... The module/subwf linting is happening for quite a while on our pipeline (I can't say when since the log are expired). But locally
pipelines lint
does lint the modules and subwf, isn't it something wanted ?
a
I don’t think it should be based on the code and your
.nf-core.yml
l
What should I do to troubleshoot the issue ?
a
Ok I figured out the reason. I am almost certain that 99% of nf-core pipelines have a
lint
section in their
.nf-core.yml
That means that their code defaults to not running linting on components when using
nf-core pipelines lint
per this line. But since
phaseimpute
has no
lint
section at all it falls into the
else
and it runs them. @mashehu you are the
git blame
for this code, was it intended that the default behaviour differed between people with and without a
lint
section, or is this actually a bit of a bug. The way I see it there are 2 potential issues here: • incorrect default behaviour of the
pipelines lint
command regarding components • bad subworkflows lint checks which aren’t working as expected for some cases
l
Ah, that's a bit of an issue 😅
I've tried by just adding
Copy code
lint:
  files_exist:
    - conf/modules.config
And all the modules/subworkflow linting dissapear... From my point of view I would prefer to have the whole linting when running the pipeline instead of having to individually lint pipeline/module/subwf 🤔
I've also discovered another "issue" when a module has an
_
in it's name. nf-core thinks it should be a folder. (e.g. I have a local module named
add_columns
and the linting give me
Included module/subworkflow add/columns missing in meta.yml
) As we shouldn't use CamelCase for the modules names how can we solve this ? I have local modules such as
bam_chr_extract
than would be difficult to name without separation...
a
Local modules should be renamed to match the nf-core style ultimately. This was a fairly recent change I think. I would prefer it if any module could use underscores but that decision was made a very long time ago in nf-core and I think there would be a lot of overhaul if we tried to change it as so much of the tools logic relies on it.
l
Make sense 🙂 I will do the change in our pipeline.
After a bit of testing
completionEmail
,
completionSummary
and others needs to be just above the workflow where they are used and the warning disappear. This might need to be moved in the TEMPLATE.