Hi. I configured a github action for linting and executing

nf-core pipelines lint

. Our

.nf-core.yml

config file in the pipeline repo has

nf_core_version: 3.2.0

,

repository_type: pipeline

and lint config specify

files_exist

,

files_unchanged

,

multiqc_config

and

nextflow_config

keys to skip some file checks and config parameter checks. We were expecting the lint functionality to execute

module_changes

test along with the existing 20+ pipeline tests (https://nf-co.re/tools/docs/latest/pipeline_lint_tests/ + https://nf-co.re/tools/docs/latest/module_lint_tests/ ). But, from the verbose output -- I see only the pipeline_lint_tests running. As per the source code @ https://github.com/nf-core/tools/blob/3.2.0/nf_core/pipelines/lint/__init__.py#L573, pipeline lint tests are a union of pipeline tests and module lint tests. So, technically

module_changes

must be included in that set. Am I missing any configuration or doing something wrong that is not triggering the

module_changes

test when pipelines lint is run? For testing it, I added a new line in the nf-core's multiqc main.nf that should theoritically trigger an error that copy the file in the current pipeline is different from the remote nf-core module that is imported. No validation error is reported. Only when

nf-core modules lint

is run, the test for

module_changes

is executed and the error reported. Please help.

Hi @Ram Anusuri, can you copy the exact content of the lint section in your

.nf-core.yml

? Is your pipeline an nf-core pipeline or a custom one?

Its a custom pipeline built using nf-core template. It has local, nf-core and an internally built ext-core modules (modules/local/<...>, modules/nf-core/multiqc, modules/ext-core/untar)

lint:
  files_exist:
    - assets/nf-core-tbwgsnpclustering_logo_light.png
    - docs/images/nf-core-tbwgsnpclustering_logo_dark.png
    - docs/images/nf-core-tbwgsnpclustering_logo_light.png
    - .github/ISSUE_TEMPLATE/config.yml
    - .github/workflows/awstest.yml
    - .github/workflows/awsfulltest.yml
    - .github/workflows/ci.yml
    - .github/workflows/linting_comment.yml
    - conf/igenomes.config
    - CODE_OF_CONDUCT.md
    - LICENSE
  files_unchanged:
    - assets/nf-core-tbwgsnpclustering_logo_light.png
    - docs/images/nf-core-tbwgsnpclustering_logo_dark.png
    - docs/images/nf-core-tbwgsnpclustering_logo_light.png
    - .github/CONTRIBUTING.md
    - .github/ISSUE_TEMPLATE/bug_report.yml
    - .github/ISSUE_TEMPLATE/config.yml
    - .github/ISSUE_TEMPLATE/feature_request.yml
    - .github/PULL_REQUEST_TEMPLATE.md
    - .github/workflows/branch.yml
    - .github/workflows/linting.yml
    - .github/workflows/linting_comment.yml
    - assets/email_template.html
    - assets/email_template.txt
    - assets/sendmail_template.txt
    - docs/README.md
    - CODE_OF_CONDUCT.md
    - LICENSE
    - .gitignore
  multiqc_config:
    - report_comment
  nextflow_config:
    - manifest.name
    - manifest.homePage

I am trying to recreate this with a new created pipeline and the test works for me

I just tried your approach. Ran pipelines lint. Did not fail. Screenshot below. Ran modules lint. It failed on error as expected. See screenshot below (ignore the error about unable to connect to container url)

could you copy the output when you run

nf-core --verbose pipelines lint

? let's see if this gives us more information 🤞🏻

Thank you for looking into this. Attached is the output log with the val(test) in place in main.nf of nf-core multiqc module

what happens if you delete the chache in

~/.nextflow/nf-core

and try again?

I tried it several times. Will give it a try again.

if this still doesn't work, could you try to add the parameter

--show-passed

?

I will attach the nf-core.yml shortly. I don't have any other nf-core yml in any subdirectories. I am running linting from the parent directory.

I can't see why is this skipped 🙈 last check, you don't have an .nf-core.yml file anywhere even outside the directory you are running the command from?

There are other projects in my workspace that contain .nf-core.yml, but they pretty much resemble or match what is in the current file. But, let me try something else then -- I will create a test repo in a temp folder or my home folder and test out the lint functionality.

are you able to install the dev version to nf-core in the machine you are running this?

Sure. I think I can do that. So, how do I get the dev version?

if you use pip, you can run this

pip install --upgrade --force-reinstall git+<https://github.com/nf-core/tools.git@dev>

Ok. Let me create a venv and do an install. Will try it out on the same nf-test-wgsnptest repo after that. It will take a little time for me get the venv setup and install it. Will get back soon.

you see the failing multiqc test?

Yes

🙌🏻 sorry for the very long debugging process 😱

image.png

then, it will be fixed with the new release of nf-core/tools

I am in the process of running linting on our production ready pipelines (via github actions). I don't think I can suggest a dev version of the nf-core tool to our clients yet. But, I will update them on this. Will see how they want me to move forward with. But looks like there is an apparent bug in the 3.2.x version of the code where it is not getting picked up even though the source code (line 573 of init.py of pipelines lint) shows it is a union of tests.