I wrote an open-source linting tool that lets you ...
# shameless-promo
a
I wrote an open-source linting tool that lets you write linting rules in python and avoid parsing the DSL: https://reftrace.com/blog/announcing-linting/ Here's an example:
Copy code
# This file should exist in the root of your pipeline directory
def has_label(directives):
    return len(directives.label) > 0

def has_cpus(directives):
    return len(directives.cpus) > 0

def rule_has_label_or_cpus(module):
    for process in module.processes:
        if not (has_label(process.directives) or has_cpus(process.directives)):
            fatal("process %s has no label or cpus directive" % process.name)
Feedback is much appreciated!
❤️ 1
a
With the new Nextflow Language Server I wonder if this ability to write custom rules could be incorporated there somehow. Separately, wonder if we could use this functionality to write more fine-grained linting within nf-core (which essentially does some very rough regex parsing of the DSL) cc @Júlia Mir
m
I think @Ben Sherman might have some clues about how the Language server can interact there
b
user-defined linting rules seems pretty useful, I just haven't thought very hard about it. RefTrace should work perfectly fine as a CI linting tool, could be a quick win for improving the nf-core linting which is already in Python
not sure how we would support custom linting rules in the language server. I suspect users would have to write a custom extension. so RefTrace seems like a better option for that now
a
I added nf-core linting rules as built-in to the tool. They can be run with:
Copy code
reft lint nfcore
Here's more info I focused on module rules, specifically container and label rules. They have full test coverage.
a
Interesting. I guess my main concern here is unification/concordance of the different lexers/syntax parsers if the two projects are separate and people want the linting but also the formatting. (I have no idea how this could be measured/benchmarked, but I would want to avoid if possible the sort of pain that can be experienced if you try to please both
mypy
and
pyright
)
a
A few observations: • No regex or string parsing of the directives was needed. I did benefit from the rich DSL representation RefTrace provides. • I tried to mirror nf-core/tools as much as possible. It was easier for me to implement the rules natively (in Go) rather than expose the exact error plumbing in python* userland. It can be done and I can see how unfamiliarity with Go could be an issue for rule authors. • RefTrace is fast -- a few secs to lint a pipeline vs a few secs per module with nf-core/tools. I imagine this is because nf-core/tools does more work and also maybe because nf-core/tools makes network calls. Thank you all for your feedback! I would be very interested to know if you try the
reft lint nfcore
functionality. I did some git surgery to remove a jar from the repo, so you have to re-clone if building from source. (sorry -- figured it was still small enough)
Interesting. I guess my main concern here is unification/concordance of the different lexers/syntax parsers if the two projects are separate and people want the linting but also the formatting.
Definitely -- I'd like for RefTrace to be conservative here. As in, it shouldn't reject programs that are actually correct. It doesn't do any formatting.
a
I would think we would want to implement the rules-set ourselves in a
rules.py
because we would want to be able to update/disable rules selectively (as we currently allow using the config yml) and would not want to rely on updates to this tool in order to update nf-core tooling. The barrier of yet another language (Go) on top of groovy/python would also be a barrier to devs making new rules easily I suspect (which is precisely the point of reftrace AFAICT!) Calling out to remote modules is definitely a necessity for some of the tests but I think much of the linting can be done without and the rest with a single clone/pull of the remote repo. Can
rules.py
contain any arbitrary Python? (Can it use third-party libraries?) It would also possibly be useful if rules could take extra arguments (must have a default) through some form of configuration as this could allow the same rule-set to be used over multiple repositories/remotes with ease. RE: the comparison to the new language server, I am more concerned that they both lex/tokenize the same code the same way. My naive hope (before this RefTrace was announced) was the language server could be used by nf-core tools to parse code in the way reftrace does here. Ideally it's best if tools are in sync (ie nf-core tools/other tooling could all depend on the language servers tokenization).
💯 1
a
I think you're right. I added proper Python bindings: docs Here's some example code using vanilla CPython:
Copy code
from reftrace import Module

module = Module("path/to/nextflow.nf")
print(f"File: {module.path}")
print(f"DSL Version: {module.dsl_version}")

for process in module.processes:
    print(f"\nProcess: {process.name}")
    for directive in process.directives:
        if directive.format == "simple":
            print(f"  Container: {directive.simple_name}")
        else:  # ternary
            print(f"  Container: {directive.condition} ? {directive.true_name} : {directive.false_name}")
and some output:
Copy code
File: /home/andrew/circrna/modules/local/annotation/main.nf
DSL Version: 2

Process: ANNOTATION
  Container: ((workflow.containerEngine == singularity) && !(task.ext.singularity_pull_docker_container)) ? https://depot.galaxyproject.org/singularity/pandas:1.5.2 : biocontainers/pandas:1.5.2
I need to flesh out the bindings a bit and publish on PyPI. And yes -- the goal of RefTrace is to make it easier for people to write correct pipelines! Hopefully having native Python access to the DSL helps. As far as integration and speeding things up, would you prefer a separate pip-installable tool or a PR to
nf-core/tools
?
a
My intention would be that we maintain our linting in nf-core using a custom rules.py. (Assuming we moved to reftrace - I do not have final say on that!)
I'll take a look at these docs in more detail tomorrow
a
It’s pretty spartan right now. I’ll have them fleshed out and an example published by tomorrow evening. Thanks in advance for checking it out.
Here's a linting example in pure python: https://github.com/RefTrace/reftrace-python-example Users write rules in rules.py :
Copy code
for process in module.processes:
        for container in process.containers:
            names = get_names(container)
            for name in names:
                if ' ' in name:
                    results.errors.append(ModuleError(
                        line=1, # TODO: use the actual line number
                        error=f"container name '{name}' contains spaces, which is not allowed"
                    ))
The core reftrace library docs and API docs are updated. I still need to do more here. It's annoying building everything from source. Once I publish the core library on PyPI, that example will run with
pip install reftrace
.