https://pantsbuild.org/ logo
Join Slack
Powered by
# plugins
  • c
    I have developed several plugins for running 
    uv sync
    and 
    uv build
    . I want to create a goal that wraps the other goals. Does anyone have a reference for how to do this? I'm finding this surprisingly difficult. Here's a ref to my current plugins: https://github.com/TechnologyBrewery/pants-uv-lifecycle-plugin/tree/dev/src/pants_uv_lifecycle_plugin
    h
    • 2
    • 12
  • l
    Hello 👋! Looking for a little guidance with a plugin I wrote internally, I think I've addressed my problem for now but hoping to get some insight into properly fixing my code (maybe I'm just approaching it all wrong to begin with). After updating from 2.21 to 2.24 (identified error from 2.23) I'm getting this error:
    Copy code
    native_engine.IntrinsicError: Get(InterpreterConstraints, InterpreterConstraintsRequest, InterpreterConstraintsRequest(addresses=Addresses([Address(src/******/__init__.py:lib), Address(src/******/app.py:lib), Address(src/******/conf.py:lib), Address(src/******/schemas.py:lib), Address(src/******/__init__.py:lib), Address(src/******/cli.py:lib), Address(src/******/core.py:lib), Addr
ess(src/******/cors.py:lib), Address(src/******/models.py:lib)]), hardcoded_interpreter_constraints=None)) was not detected in your @rule body at rule compile time. Was the `Get` constructor called in a non async-function, or was it inside an async function defined after the @rule? Make sure the `Get` is defined before or inside the @rule body.
    w
    • 2
    • 34
  • h
    Is there a recommended way to create a common 1st party library that's shared between plugins?
    h
    • 2
    • 8
  • h
    Another question unrelated to the above Is there a way to run a goal from another goal? Specifically, I'm looking at conditionally (re-)generating a lockfile as part of another plugin
    c
    • 2
    • 2
  • i
    I'm having trouble just getting off the ground when writing my plugin. Generally I'm trying to install a Go tool and then utilize it to generate some code. However I can't really get past even downloading the sources. More details in the thread.
    b
    • 2
    • 7
  • c
    I'm trying to make a plugin that makes a Docker-wrapped Pex deterministic. Pex will happily timestamp all the files inside the 
    .pex
    file from the 
    SOURCE_DATE_EPOCH
    envvar correctly. However, the timestamp of the 
    .pex
    file itself will always update. 
    COPY
    -ing it in a Dockerfile will therefore create a new sha every time, making it uncachable. I need to somehow fix up the timestamp of the 
    .pex
    file post-pex-build and pre-docker-build within the 
    package
    goal. My problem is that I don't even know how I'd approach this problem. Any ideas? (Convincing Pex to timestamp the file correctly would of course also fix the problem.)
    g
    c
    • 3
    • 9
  • g
    Has anyone here played around with building a plugin/custom goals for "local workflows"? We have a fairly sizeable collection of test-runners, utility scripts, etc, that wrap pants, but it's kinda awkward and fiddly. Instead one of my colleagues asked if we could have a prefix like 
    pants the-repo ...
    . It seems technically doable, but I'm not 100% if it's wise.
    f
    e
    +2
    • 5
    • 11
  • b
    Hi there, I've successfully written a pants plugin to set custom docker image tags if an env var is present during 
    pants package
    . Now I'd like to test it using the 
    RuleRunner
    setup. Since I'm completely new to writing custom pants code I'm having a hard time debugging the error message. Any help is much appreciated, thanks! 🙏 😊 See code in 🧵
    w
    • 2
    • 10
  • g
    Should 
    migrate-call-by-name
    work on standalone plugins? I'm trying to migrate my plugins now but it fails out immediately:
    Copy code
    $ pants --source-root-patterns='["pants-plugins"]' migrate-call-by-name pants-plugins/::
13:00:06.23 [ERROR] '/home/ts/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.24.0/lib/python3.9/site-packages/pants/option/subsystem.py' is not in the subpath of '/home/ts/Repositories/pants-backends' OR one path is relative and the other is absolute.
    w
    • 2
    • 9
  • a
    Is there a canonical example of: • Running the bulk (all of) your product code in an (old) version of python • Running your pants plugin code w/ whatever version of python Pants requires (3.9 prior to 2.25, 3.11 after) • Running python codegen plugin code against the version of python that your product code needs • running pytest and mypy against it all We kinda got lucky that our vendor required Py3.9 But they upgraded to py3.10 which pants completely skipped. And now we're struggling with resolves and interpreter constraints and stuff
    b
    • 2
    • 3
  • a
    Not sure how to even ask this question, but suppose I have a backend with an 
    experimental-deploy
    goal. That goal runs a python file in a sandbox. How can a user of my backend pass command line arguments to the script? I did try just running 
    pants experimental-deploy src/mything -- --foo=bar --baz=quux
    but those flags don't make it through to sys.argv in the process, which makes sense. My deploy rule is just constructing a Process and wrapping it up as a DeployProcess, but I'm unclear about how I would obtain additional, arbitrary, command line arguments for that Process. Is this a thing?
    g
    • 2
    • 3
  • a
    @fast-nail-55400 What's the deal with 
    AdhocProcessRequest
    and 
    AdhocProcessResult
    - what's different about them vs regular process and process result? Trying to build an adhoc packaging tool to do some filepath manipulation before the packaged output of a dependecy is passed downstream. As far as I can tell, there's no existing way to do this.
    f
    • 2
    • 11
  • e
    is it possible to batch/partition arbitrary rules, not just things like test/lint/fix goals? if so, is there a good example in core that someone can point out to me?
    f
    • 2
    • 13
  • e
    e.g. let's say i have a bunch of targets, and i want to run a rule on some number of them in parallel
  • e
    doesn't have to be a target, whatever input
  • w
    Hello ✋ Can I install pants as a python package somehow so that pycharm can see its modules and make hints. Now it just says "Unresolved reference 'pants' "
    e
    • 2
    • 5
  • c
    Hi, I'm attempting to implement a couple of targets/rules that implement basic codegen (generate a JSON file from a YAML source, etc.). I've got the basics working, but I'm not sure how I can "chain" the codegen targets. Here's an illustrative BUILD file:
    Copy code
    yaml_file(
    name="my_yaml",
    source=http_source(
        "<https://raw.githubusercontent.com/codefresh-io/yaml-examples/refs/heads/master/codefresh-build-1.yml>",
        len=197,
        sha256="4f0f073a576fc44d1ad670bf886fb24f394f63b6eabb7677eb197d427f5db7b0",
    ),
    convert_to_json=True
)

json_file(
    name="my_json",
    source="codefresh-build-1.json",
    dependencies=[":my_yaml"]
)
    The 
    yaml_file
    target works fine. The rule is of the form
    Copy code
    @rule
async def generate_yaml_from_yaml_source(
    request: GenerateYAMLSourcesRequest,
) -> GeneratedSources:
    Where 
    GenerateYAMLSourcesRequest
    is subclass of 
    GenerateSourcesRequest
    .     It's when I try to trigger the generation for the 
    json_file
    target that the issue occurs. The target is also backed by a rule based on a subclass of 
    GenerateSourcesRequest
     , and in addition has code to resolve the dependencies and provide them as a snapshot during the generation. The code fails, seemingly before my rule is triggered, with the following error:
    Copy code
    native_engine.IntrinsicError: Unmatched glob from src/python/libs/portland/connectors/my_connector:my_json's `source` field: "src/python/libs/portland/connectors/my_connector/codefresh-build-1.json"
    What's the appropriate way to chain these codegen steps - with the caveat that the 
    json_file
    target won't always be used in a chain, it should also be used independently.
    f
    • 2
    • 6
  • h
    I'm bumping into a silly issue with adding custom requirements to plugins.
    Copy code
    pants.toml

[GLOBAL]
pants_version = "2.26.0"
pythonpath = ["%(buildroot)s/pants-plugins"]
backend_packages = [
    "pants.backend.python",
    "pants.backend.plugin_development",
    "myplugin",
]


[python]
interpreter_constraints = ["==3.9.*"]
enable_resolves = true


[python.resolves]
pants-plugins = "pants-plugins/lock.json"
    Copy code
    pants-plugins/BUILD

pants_requirements(resolve='pants-plugins')

python_requirement(
    requirements=['requests'],
    resolve='pants-plugins',
    name='requests',
)
    Copy code
    pants-plugins/myplugin/register.py

import requests

def rules():
    return []
    Running basically any 
    pants
    command at this point I get
    Copy code
    ModuleNotFoundError: No module named 'requests'
    What am I doing wrong? I'm pretty sure that I'm following all the instructions to the letter...
    g
    • 2
    • 3
  • h
    Is it possible to run a process (specifically 
    PexProcess
    ) that doesn't immediately quit? I'm looking into implementing the "serve" option for `mkdocs`(link) so it needs to remain open until explicitly stopped
    h
    f
    • 3
    • 11
  • h
    Is there a way to set a mandatory option, when not provided will raise a generic error? It can be tricky to validate all inputs, because some of them are not consumed by the goal rule...
    a
    • 2
    • 3
  • a
    I now have a few useful flags that are also docker build_args . if i wanted to avoid having to run: pants package --flag-1=X --docker-build-args='["flag-1=X"]' I would need a @rule that took in my flag (my subsystem, reall) and outputed... What? DockerBuildArgs.extended?
    Copy code
    @rule
def resolve_build_args(args: DockerBuildArgs, sub: MySubSystem) -> DockerBuildArgs:
  return args.extended(["flag-1=X"])
    h
    f
    • 3
    • 11
  • p
    I feel like this should be obvious but I'm stumped. When adding a new 
    PythonToolBase
    subsystem (in the pants repo), how do you generate the initial lockfile? I tried adding my new subsystem to 
    build_support/bin/generate_builtin_lockfiles.py
    and then running 
    pants run build-support/bin/generate_builtin_lockfiles.py -- --debug elfdeps
    (where elfdeps is the option scope of my new subsystem). But it complains that the lockfile can't be found (I know. That's why I asked to generate it.). Using 
    --keep-sandboxes
    is not helpful because of the secondary sandbox that gets created by the 
    generate_builtin_lockfiles.py
    script. Do I need to copy some other random lockfile and use it as a seed?
    ✅ 2
    • 1
    • 3
  • f
    Hi everyone. Is there by any chance an updated version of https://www.pantsbuild.org/stable/docs/writing-plugins/common-plugin-tasks/add-codegen anywhere? I'm having some issues, and at least part of it seems to be outdated class names in the doc.
    g
    • 2
    • 31
  • f
    Next question: I'm generating the code from a process. That might fail (for whatever reason, be it transient hardware issue or user typed error in the source). What's the correct way to pass that failure along the call stack? Currently I have this, but the build just continues on merrily other than the console output.
    Copy code
    output_snapshot = []
    if result.exit_code == 0:
        <http://logger.info|logger.info>(f"✓ Generated {output_file_name} from {source_file_name}")
        output_snapshot = await Get(Snapshot, Digest, result.output_digest)
    else:
        logger.error(f"✗ Failed to generate {output_file_name}: {result.stderr}")
        output_snapshot = await Get(Snapshot, Digest, EMPTY_DIGEST)
    
    return GeneratedSources(output_snapshot)
    f
    g
    • 3
    • 7
  • w
    Hello all, I have a quick question with how to use your 
    adhoc_tool
    to run 
    tach
    on a code base. Suppossing I have a BUILD file
    Copy code
    pex_binary(
    name="tach",
    entry_point="tach",
    dependencies=[
        "python-default#actually_shared",  # resolves Tach from PyPI
    ],
)

adhoc_tool(
    name="tach_check",
    runnable=":tach",
    args=[
        "check",        # Run `tach check`
        ".",            # On the whole repo (or a subdir if you want)
    ],
    log_output=True,
    output_directories=[],
    execution_dependencies=[
        ":src",         # Source code
        "tach.toml",     # Config file
    ],
    root_output_directory=".",
)
    Point to that with 
    check-tach = "run pants-plugins/cre/tach:tach_check"
    , When i run 
    pants check-tach ::
    , i get a list of all sub dirs and a question as follows:
    Copy code
    * src/tools/sales/ingest.py
  * src/tools/sales/ingest_stages/__init__.py
  * src/tools/sales/ingest_stages/r2s.py
  * src/tools/sales/ingest_stages/s2m.py
  * src/tools/sales/ingest_stages/s2m_test.py:tests
  * src/tools/sales/load_data.py
  * src/tools/sales/plants/__init__.py
  * src/tools/sales/plants/adr.py
  * src/tools/sales/plants/adr_test.py:tests
  * src/tools/security/review.py
  * src/tools/venv:create

Please select one of these targets to run.
    I am trying to run 
    tach check .
    , what am i doing wrong? Any help would be greatly appreciated on this, thanks in advance
    f
    • 2
    • 3
  • a
    Related question ☝️ if I have a linter that operates at the repository level, not the target or file level, is that an appropriate thing to use a pants plugin for? Looking at the lint goal, I see a LintTargetsRequest and a LintFilesRequest, which makes sense in the context of Pants' model. Before I go and subclass AbstractLintRequest, is this just a perverse thing to do in pants? For context, tach is a tool for checking that module dependency rules are not violated. I know that I can do this through Pants itself, but the DX is a little painful. 
    tach check
    uses a toml file to check the dependencies of declared modules, and make sure that we haven't accidentally imported a gajillion ML libraries into the wrong place. It would be nice to have it work under 
    pants check
    .
    g
    • 2
    • 18
  • f
    Question on the rule graph. I've made a plugin that can convert 
    .foo
    files into 
    .bar
    format. It does dependency inference and everything. Now I've written a second plugin that converts 
    .bar
    files into 
    .baz
    files. The 
    .foo
    files are my primary sources. But when I try to HydrateSources with enable_codeen and the for_sources_type set to 
    .baz
    , none of the rules are invoked, nor are the dependency inference functions. What is the secret to get it to chain the two rules together?
    f
    • 2
    • 10
  • l
    I've recently published a plugin for running basedpyright type checking during 
    pants check
    to avoid managing Node.js for pure python projects. https://github.com/jacoblearned/pants-basedpyright I'd like to add optional support for its baseline feature which would require writing the output files from a 
    VenvPexProcess
    back to disk, but it seems there's no way to get a handle to the 
    workspace
    during 
    CheckRequest
    executions to write the baseline file safely (for good reason). Is my main option here to implement a custom 
    @goal_rule
    to run separately from 
    pants check
    that collects output digests from 
    CheckResults
    and then write those to disk?
    w
    • 2
    • 3
  • f
    We're working on setting up a build flow that involves multiple pipelined code-generation steps. I've created a series of examples of the types of flows we have, just using C code and gcc as stand-ins for our various code generators. Some of these patterns require inferring dependencies from reading the generated files. We'd love it if someone could take a look and help us figure out the most elegant way to implement these flows in pants. https://github.com/NVIDIA/Dependency-Patterns
  • s
    Do plugins support call-by-name in the latest release? I recall seeing a note about wanting to deprecate call-by-type soon.
    w
    f
    • 3
    • 10