cool-easter-32542
04/12/2024, 9:30 PM@rule
in the module, look up its solution in the rule graph.Get
in a @rule
, with its corresponding call-by-name syntax (using whatever code-rewriting API is best maintained currently)
EDIT: #20574 implemented items 2 and 3.
pantsbuild/pantscool-easter-32542
04/13/2024, 2:39 PMhelm_chart
target does not support any of those kind of files at the moment, for a more complete implementation, support for these files should be added: values.schema.json
, README.md
, LICENSE
and templates/NOTES.txt
.
Describe the solution you'd like
Add one optional single-source field per file to the helm_chart
. This would allow to later provide with add-on implementations that could codegen each of those files independently from different sources.
Describe alternatives you've considered
N/A
Additional context
Helm documentation describing chart file structure:
https://helm.sh/docs/topics/charts/
pantsbuild/pantscool-easter-32542
04/14/2024, 11:50 PMpsutil==5.9.0
dependency was being built to a wheel incorrectly on Apple Silicon: the native code was ending up as x86-64, rather than arm64. There's two layers to this bug:
1. the CI steps attempt to set ARCHFLAGS='-arch arm64'
but these seemed to stop working in release_2.17.0.dev5...release_2.17.0a0 (see #20765 (comment) for analysis)
2. the runner is defaulting to i386/x86-64 shell (etc.) by default, rather than arm64 (a freshly installed self-hosted runner on my M1 laptop runs in arm64 shells by default: see #20765 (comment) for analysis)
Fixing 2 is arguably the better fix, as then the runner will be doing the right thing by default. If we fix this via 1, we need to make sure all uses of this runner apply the flags when they build things (this includes in scie-pants).
This can be confirmed by running commands like:
ā¢ arch
=> outputs i386
, should output arm64
ā¢ echo 'int main() { return 0; }' > test.c && clang test.c -o test && file test
(compile some C code) => outputs test: Mach-O 64-bit executable x86_64
, should output ... arm64
Pants version
2.17.0a0
OS
arm64 macOS in Ci
Additional info
N/A
pantsbuild/pantscool-easter-32542
04/15/2024, 2:51 AMlayout
setting could be added, which set a default compression value for internal use:
pants/src/python/pants/backend/python/util_rules/pex.py
Lines 196 to 198 in </pantsbuild/pants/commit/902d3acb84c1b64c53c05e882200e4d578884c03|902d3ac>
pantsbuild/pantscool-easter-32542
04/15/2024, 11:44 AMpants generate-lockfiles
when a jvm_artifact
contains a jvm_exclude
that only specifies a group will fail with a "Failed to parse [group-name]" message from Coursier. This is contrary to the documentation for jvm_exclude
which states "`jvm_exclude`: Exclude the given artifact
and group
, or all artifacts from the given group
."
Pants version
2.20.0rc2
OS
MacOS
Additional info
Example Repo https://github.com/NGustafson/pants-examples/blob/main/3rdparty/jvm/BUILD
This repo has a single jvm_artifact with nothing else configured. Attempting to run pants generate-lockfiles
will cause this error:
pants generate-lockfiles
[ERROR] 1 Exception encountered:
Engine traceback:
in `generate-lockfiles` goal
ProcessExecutionFailure: Process 'Running `coursier fetch` against 1 requirement: org.slf4j:slf4j-log4j12:2.0.12' failed with exit code 1.
stdout:
stderr:
+ coursier_exe=__coursier/./cs-aarch64-apple-darwin
+ shift
+ json_output_file=coursier_report.json
+ shift
++ pwd
+ working_dir=/private/var/folders/cm/gmrdwxcn7tv_cct4dzg38w91kjyl1q/T/pants-sandbox-aM4FVB
+ __coursier/./cs-aarch64-apple-darwin fetch -r=<https://maven-central.storage-download.googleapis.com/maven2> -r=<https://repo1.maven.org/maven2> --no-default --json-output-file=coursier_report.json org.slf4j:slf4j-log4j12:2.0.12 --local-exclude-file PANTS_RESOLVE_EXCLUDES
Failed to parse org.slf4j
Failed to parse org.slf4j
pantsbuild/pantscool-easter-32542
04/16/2024, 2:52 PM./pants export
currently follow Pex's default of generating "hermetic" console scripts, by rewriting all the shebangs of scripts under bin/
to pass -sE
to python
. This prevents the use of exported scripts with a custom PYTHONPATH
, which breaks use-cases like running pylint
with in-repo plugins and a custom source root.
Describe the solution you'd like
Pex 2.1.118 added a --non-hermetic-scripts
option to the venv
tool, which disables the addition of -sE
to script shebangs. Pants could pass that flag when running the export logic.
This should definitely be safe to do when exporting a non-symlinked venv. I'm not sure if it would be safe to do in the symlink case, since the virtualenv might also be used for other processes within Pants. Maybe this should be an option on the export
goal-subsystem, with the caveat that if you enable it you will get worse caching performance out of the symlink mode.
Describe alternatives you've considered
I thought about writing a wrapper script around ./pants export
that re-re-writes the shebangs to remove -sE
. I worry about the safety of doing this when using symlinked exports, since I'd be munging files within the Pants cache.
To avoid munging data in the pants cache, the wrapper script would need to do something like:
1. Run ./pants export
with symlinking
2. Delete the top-level symlink under dist/export
3. Recreate the directory structure of a virtualenv under dist/export
in place of the deleted link
4. Within the virtualenv skeleton, link back to everything in the Pants-cache venv except for bin/
5. Copy all the files from Pants-cache bin/
to the virtualenv under dist/export
, rewriting shebangs along the way
I'm confident I could write such a script, but don't really want to if it turns out it's safe for Pants to always run with --non-hermetic-scripts
š
Additional context
The pylint
example might seem unrealistic since we'd typically want people to run ./pants lint
, but it is important for IDE integrations. We have other custom tools that do wacky things with importlib
that are broken in the same way.
pantsbuild/pantscool-easter-32542
04/16/2024, 6:19 PM[docker.registries.registry-name]
address = "{build_args.CONTAINER_REGISTRY}"
but it looks like address string doesn't perform variable interpolation. This is in contrast to e.g. repository
, which allows you to interpolate e.g. repository = "{name}"
.
pantsbuild/pantscool-easter-32542
04/17/2024, 12:50 AMpants.toml
to have `pants_version = "2.20.0"`: https://github.com/pantsbuild?q=example&type=all&language=&sort=name
āļø https://github.com/pantsbuild/example-adhoc
āļø https://github.com/pantsbuild/example-codegen
āļø https://github.com/pantsbuild/example-django
āļø https://github.com/pantsbuild/example-docker
āļø https://github.com/pantsbuild/example-golang
āļø https://github.com/pantsbuild/example-javascript
āļø https://github.com/pantsbuild/example-jvm
āļø https://github.com/pantsbuild/example-kotlin
āļø https://github.com/pantsbuild/example-python
āļø https://github.com/pantsbuild/example-serverless
āļø https://github.com/pantsbuild/example-visibility
(NB. not https://github.com/pantsbuild/example-plugin)
Bonus points (repos that use pants but aren't examples):
āļø https://github.com/pantsbuild/scie-pants
ā https://github.com/pantsbuild/setup
pantsbuild/pantscool-easter-32542
04/17/2024, 12:52 AMpants.toml
to have `pants_version = "2.19.0"`: https://github.com/pantsbuild?q=example&type=all&language=&sort=name
ā https://github.com/pantsbuild/example-adhoc
ā https://github.com/pantsbuild/example-codegen
ā https://github.com/pantsbuild/example-django
ā https://github.com/pantsbuild/example-docker
ā https://github.com/pantsbuild/example-golang
ā https://github.com/pantsbuild/example-javascript
ā https://github.com/pantsbuild/example-jvm
ā https://github.com/pantsbuild/example-kotlin
āļø https://github.com/pantsbuild/example-python (pantsbuild/example-python#133)
ā https://github.com/pantsbuild/example-serverless
ā https://github.com/pantsbuild/example-visibility
(NB. not https://github.com/pantsbuild/example-plugin)
Bonus points (repos that use pants but aren't examples):
ā https://github.com/pantsbuild/scie-pants
ā https://github.com/pantsbuild/setup
pantsbuild/pantscool-easter-32542
04/17/2024, 12:53 AM2.20.x
milestone.
ā¢ Semi-regularly check https://github.com/pantsbuild/pants/milestone/63 to have an understanding of what's blocking (and any others that are still open, e.g. an issue in 2.19.x might require fixing for 2.20.x). NB. we don't seem to have an explicit mechanism to indicate that something is a blocking issue.
ā¢ Check/follow-up on #development on slack as required.
āļø Prepare release assets:
ā¢ a 2.20.x.md
"what's new", similar to https://github.com/pantsbuild/pants/blob/main/src/python/pants/notes/2.19.x.md
ā¢ a blog posts similar to pantsbuild/pantsbuild.org#134
āļø Regular release candidate releases as usual (done by the MOTW, usually), and encourage people to test them
āļø Publish the release when ready
āļø When published, update example repos similar to #20526
(To be clear, usually others have done the work, but I've been trying to have the overview to see if anything falls through the cracks, and prompt things like "it seems like we're close to being ready".)
pantsbuild/pantscool-easter-32542
04/17/2024, 3:07 PMjq
).
However, in my opinion, this mostly makes sense if the "fields" will help performance by not performing the associated peek
goals/rules - as otherwise we're scope creeping a poor man's jq
into our CLI which feels excessive.
pantsbuild/pantscool-easter-32542
04/17/2024, 10:26 PMimport sqlalchemy
import greenlet
def handler():
print(sqlalchemy.__version__)
print(greenlet.__version__)
Running pants package :cloud_function
gives me the following build error
A distribution for greenlet could not be resolved for /home/jackevans/.cache/nce/67912efc04f9156d8f5b48a0348983defb964de043b8c13ddc6cc8a002f8e691/cpython-3.9.18+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9.
Found 1 distribution for greenlet that do not apply:
1.) The wheel tags for greenlet 3.0.3 are cp311-cp311-manylinux_2_28_x86_64, cp311-cp311-manylinux_2_24_x86_64 which do not match the supported tags of /home/jackevans/.cache/nce/67912efc04f9156d8f5b48a0348983defb964de043b8c13ddc6cc8a002f8e691/cpython-3.9.18+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9:
cp39-cp39-manylinux_2_38_x86_64
... 809 more ...
If I use
resource(name="platforms", source="platforms.json")
Instead of
file(name="platforms", source="platforms.json")
Inside the following BUILD
python_sources(name="root")
python_requirement(name="sqlalchemy", requirements=["sqlalchemy"])
python_requirement(name="greenlet", requirements=["greenlet"])
python_google_cloud_function(
name="cloud_function",
complete_platforms=[":platforms"],
handler="app.py:handler",
type="http",
)
resource(name="platforms", source="platforms.json")
[GLOBAL]
pants_version = "2.20.0"
backend_packages = [
"pants.backend.python",
"pants.backend.google_cloud_function.python",
]
[python]
interpreter_constraints = ['==3.11.*']
IMO attempting to mistakenly use a resource
instead of a file
when loading complete_platforms
should throw an error or at least show a warning when building python_google_cloud_function
targets.
The complete_platforms
docs state
Complete platforms should be addresses of file targets that point to files that contain complete platform JSON as described by Pex (https://pex.readthedocs.io/en/latest/buildingpex.html#complete-platform).But would be nice if this could be turning into a warning/error. Pants version 2.20.0 but also tested on 2.19* OS Both! Additional info An equivalent
pex_binary
target that uses the same complete_platform
resource works. The following builds with no issues (despite :platforms
being a resource
).
pex_binary(
name="main",
entry_point="app.py",
complete_platforms=[":platforms"],
)
pantsbuild/pantscool-easter-32542
04/17/2024, 11:13 PMentry_point
refers to a known file path that is ambiguous, such as if a file is owned by a parametrised target. This warning appears to be:
ā¢ shown for existing working PEXes
ā¢ unsilenceable
08:53:11.79 [WARN] The pex_binary target //:ambiguous has the field `entry_point='./example.py'`, which maps to the Python module `example`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['//:src@tags=a', '//:src@tags=b'].
Please explicitly include the dependency you want in the `dependencies` field of //:ambiguous, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.
Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/2.20/docs/using-pants/troubleshooting-common-issues#import-errors-and-missing-dependencies>.
08:53:11.79 [WARN] Pants cannot resolve the entrypoint for the target //:ambiguous. The entrypoint EntryPoint(module='./example.py', function=None) might refer to the following:
* //:src@tags=a
* //:src@tags=b
The first warning is visible in 2.19 and earlier, while the second is new to 2.20.
Having diagnostics is good as silently failing to infer means the PEX doesn't contain the code people expect, but, unfortunately, it seems to be impossible silence the second one, by either disambiguating it properly, providing dependencies or just overruling the warning for the specific PEX (i.e. tell Pants "I know, it's fine"):
ā¢ attempts to disambiguate by specifying the target name like entry_point="example.py:src@tags=a
or entry_point="//:src@tags=a"
(as implied by the error message) result in an invalid entry point
ā¢ adding the required dependency explicitly still shows the second warning (but this does silence the first warning), and I don't think there's a way to squash unowned-dependency warnings for a single target, only globally?
As a resolution:
ā¢ potentially these two warnings are redundant, and we should only have one, maybe the first one?
ā¢ the second one should behave like the first and be silenced with a manual dependency specification
Reproducer that sets up three variations of this sort of pex_binary
and runs them (not just package, so that we see if we got a working PEX):
cd $(mktemp -d)
cat > pants.toml <<EOF
[GLOBAL]
pants_version = "2.20.0"
backend_packages = [
"pants.backend.python",
]
[python]
interpreter_constraints = ["CPython==3.10.*"]
EOF
cat > BUILD <<EOF
python_source(name="src", source="example.py", tags=parametrize(a=["a"], b=["b"]))
pex_binary(name="ambiguous", entry_point="./example.py")
pex_binary(name="with-target", entry_point="./example.py:src@tags=a")
pex_binary(name="with-deps", entry_point="./example.py", dependencies=[":src@tags=a", "!:src@tags=b"])
EOF
echo 'print("worked!")' > example.py
# force the builds to run fresh
export PANTS_LOCAL_STORE_DIR=$(mktemp -d)
set -x
# BUG:
# Two warnings:
# 1. `The pex_binary target //:ambiguous has the field ...`
# 2. `Pants cannot resolve the entrypoint ...`
# Fails to run: `ImportError: No module named example`
pants run :ambiguous
# BUG:
# Same two warnings.
# Fails to run: `ValueError: Invalid entry point specification: run = example:src@tags=a.`
pants run :with-target
# BUG:
# Just warning 2
# Runs successfully.
pants run :with-deps
# Comparison to 2.19:
# OKAY: just warning 1, fails to run
PANTS_VERSION=2.19.0 pants run :ambiguous
# OKAY: no warnings, runs successfully
PANTS_VERSION=2.19.0 pants run :with-deps
Output of pants run :ambiguous
(note the two warnings):
09:00:38.64 [WARN] The pex_binary target //:ambiguous has the field `entry_point='./example.py'`, which maps to the Python module `example`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['//:src@tags=a', '//:src@tags=b'].
Please explicitly include the dependency you want in the `dependencies` field of //:ambiguous, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.
Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/2.20/docs/using-pants/troubleshooting-common-issues#import-errors-and-missing-dependencies>.
09:00:38.64 [WARN] Pants cannot resolve the entrypoint for the target //:ambiguous. The entrypoint EntryPoint(module='./example.py', function=None) might refer to the following:
* //:src@tags=a
* //:src@tags=b
09:00:40.84 [INFO] Starting: Building local_dists.pex
09:00:42.59 [INFO] Completed: Building local_dists.pex
09:00:42.59 [INFO] Starting: Building ambiguous.pex
09:00:44.25 [INFO] Completed: Building ambiguous.pex
Traceback (most recent call last):
...
ImportError: No module named example
Output of `pants run :with-target`:
09:05:06.67 [WARN] The pex_binary target //:with-target has the field `entry_point='./example.py:src@tags=a'`, which maps to the Python module `example`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['//:src@tags=a', '//:src@tags=b'].
Please explicitly include the dependency you want in the `dependencies` field of //:with-target, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.
Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/2.20/docs/using-pants/troubleshooting-common-issues#import-errors-and-missing-dependencies>.
09:05:06.67 [WARN] Pants cannot resolve the entrypoint for the target //:with-target. The entrypoint EntryPoint(module='./example.py', function='src@tags=a') might refer to the following:
* //:src@tags=a
* //:src@tags=b
09:05:06.67 [INFO] Starting: Building with-target.pex
09:05:07.95 [INFO] Completed: Building with-target.pex
Traceback (most recent call last):
...
File "/Users/huon/.pex/unzipped_pexes/b5e9f44dc6235d1d08264c298fe901a5320f636f/.bootstrap/pex/dist_metadata.py", line 933, in parse
raise ValueError("Invalid entry point specification: {spec}.".format(spec=spec))
ValueError: Invalid entry point specification: run = example:src@tags=a.
Output of `pants run :with-dips`:
09:05:08.40 [WARN] Pants cannot resolve the entrypoint for the target //:with-deps. The entrypoint EntryPoint(module='./example.py', function=None) might refer to the following:
* //:src@tags=a
* //:src@tags=b
09:05:08.41 [INFO] Starting: Building with-deps.pex
09:05:09.67 [INFO] Completed: Building with-deps.pex
worked!
Pants version
2.20
OS
macOS
Additional info
I think the new "Pants cannot resolve the entrypoint" warning was added in #20390. Presumably it was added because the existing warning wasn't appearing in some cases?
pantsbuild/pantscool-easter-32542
04/18/2024, 12:24 AMshell_command
- but would be nice to have something built into the pipeline. Not sure how common a use case or request it is.
Describe the solution you'd like
I don't have one yet - I will prototype some possible solutions around extending the archive
target with encryption fields, but the idea is that the output of archive
or some sort of "final" output gets piped through an optional symmetric encryption block (or asymmetric if we're in the mood).
For now, the fields could be as simple as encryption type, key sizes, etc....
pantsbuild/pantscool-easter-32542
04/18/2024, 12:30 AMPython 3.7+ discoverable on your PATH.More accurately, this should read "Python 3.7, 3.8, or 3.9 discoverable on your PATH" pantsbuild/pants
cool-easter-32542
04/18/2024, 12:33 AMjavascript
backend on 3-4 python files, I tried to format but got suck here:
āŗ oss/pants-pyright % ./pants --changed-since=origin/main fmt ā 17557-npx-dep-version*
ā 107.83s Fetching with coursier: org.scala-lang:scala-reflect:2.13.7
ā ¤ 107.73s Fetching with coursier: org.jline:jline:3.20.0
ā ¤ 107.73s Fetching with coursier: org.scalameta:scalafmt-interfaces:3.2.1
ā ¤ 107.73s Fetching with coursier: org.scala-lang.modules:scala-parallel-collections_2.13:1.0.4
ā ¤ 107.73s Fetching with coursier: org.scala-lang:scala-compiler:2.13.7
Note: --changed-since=origin/main
is my fork, so it's up in sync with my branch, except for those 3-4 files.
On running this yesterday, I cancelled after about 3600 seconds, and today after 2400 seconds. As far as I know, none of this should even be running in the first place (at least, I would hope).
This is not a new problem, I've had it happen several times in the past few months.
Environment
'c.
,xNMM. -----------------
.OMMMMo OS: macOS 13.0.1 22A400 x86_64
OMMM0, Host: Macmini8,1
.;loddo:' loolloddol;. Kernel: 22.1.0
cKMMMMMMMMMMNWMMMMMMMMMM0: Uptime: 5 days, 11 hours, 2 mins
.KMMMMMMMMMMMMMMMMMMMMMMMWd. Packages: 172 (brew)
XMMMMMMMMMMMMMMMMMMMMMMMX. Shell: zsh 5.8.1
;MMMMMMMMMMMMMMMMMMMMMMMM: Resolution: 2560x1080
:MMMMMMMMMMMMMMMMMMMMMMMM: DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX. WM: Quartz Compositor
kMMMMMMMMMMMMMMMMMMMMMMMMWd. WM Theme: Blue (Dark)
.XMMMMMMMMMMMMMMMMMMMMMMMMMMk Terminal: Apple_Terminal
.XMMMMMMMMMMMMMMMMMMMMMMMMK. Terminal Font: SFMono-Regular
kMMMMMMMMMMMMMMMMMMMMMMd CPU: Intel i5-8500B (6) @ 3.00GHz
;KMMMMMMMWXXWMMMMMMMk. GPU: Intel UHD Graphics 630
.cooc,. .,coo:. Memory: 17866MiB / 32768MiB
pantsbuild/pantscool-easter-32542
04/18/2024, 1:01 PMcat << EOF >pants.toml
[GLOBAL]
pants_version = "2.20.0"
backend_packages = [
"pants.backend.python",
]
[python]
enable_resolves = true
interpreter_constraints = ["==3.11.*"]
[python.resolves]
python-default = "3rdparty/python/lock.txt"
[python-repos]
indexes = [
"<https://pypi.org/simple/>",
"<https://download.pytorch.org/whl/cpu>",
]
EOF
cat << EOF >BUILD
python_requirement(
name="torch_cuda",
requirements=[
"torch==2.2.1,!=2.2.1+cpu",
],
)
EOF
pants generate-lockfiles
Actual output:
14:36:40.10 [ERROR] 1 Exception encountered:
Engine traceback:
in `generate-lockfiles` goal
ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:
stderr:
Expected sha256 hash of 26bd2272ec46fc62dcf7d24b2fb284d44fcb7be9d529ebf336b9860350d674ed when downloading torch but hashed to b90669b162984e302fbd05c9c270ef796e467903944ecefa7457babe9611607e.
Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
Apparently, torch-2.2.1-cp39-none-macosx_11_0_arm64.whl
has different content on both indexes.
$ sha256sum torch-2.2.1-cp39-none-macosx_11_0_arm64.whl # PyPI version
26bd2272ec46fc62dcf7d24b2fb284d44fcb7be9d529ebf336b9860350d674ed torch-2.2.1-cp39-none-macosx_11_0_arm64.whl
$ sha256sum torch-2.2.1-cp39-none-macosx_11_0_arm64.whl # PyTorch repository version
b90669b162984e302fbd05c9c270ef796e467903944ecefa7457babe9611607e torch-2.2.1-cp39-none-macosx_11_0_arm64.whl
Pants version
v2.20.0
OS
Linux
Additional information
I suspected the PEX upgrade as part of the Pants upgrade to be the culprit, so I started testing different PEX versions. I found that PEX v2.1.148 creates the lockfile as expected, whereas v2.1.149, v2.1.150, and v2.3.1 don't. The diff look unsuspicious to me, though.
Related to #18965.
pantsbuild/pantscool-easter-32542
04/18/2024, 1:31 PMcool-easter-32542
04/18/2024, 2:31 PMBoolOption
type says:
If you don't provide avalue, this becomes a "tri-bool" where the property will returndefault
This is consistent with the other "scalar" option types, however in practiceif unset by the user.None
BoolOption
does not adhere to this behavior, while the others do.
Given this example subsystem class:
from pants.option.option_types import BoolOption, StrOption, IntOption, FloatOption
from pants.option.subsystem import Subsystem
class MySubsystem(Subsystem):
name = "My subsystem"
options_scope = "my-subsystem"
help = "My subsystem"
bool_option = BoolOption(
help="a boolean",
default=None,
)
int_option = IntOption(
help="an integer",
default=None,
)
float_option = FloatOption(
help="a float",
default=None,
)
str_option = StrOption(
help="a string",
default=None,
)
We can see in the following help output that the default (and current) value for the BoolOption
is False
, not None
as documented/expected:
$ pants --no-pantsd help my-subsystem
`my-subsystem` subsystem options
--------------------------------
My subsystem
Activated by zocdoc.pants
Config section: [my-subsystem]
--[no-]my-subsystem-bool-option
PANTS_MY_SUBSYSTEM_BOOL_OPTION
bool_option
default: False
current value: False
a boolean
--my-subsystem-int-option=<int>
PANTS_MY_SUBSYSTEM_INT_OPTION
int_option
default: None
current value: None
an integer
--my-subsystem-float-option=<float>
PANTS_MY_SUBSYSTEM_FLOAT_OPTION
float_option
default: None
current value: None
a float
--my-subsystem-str-option=<str>
PANTS_MY_SUBSYSTEM_STR_OPTION
str_option
default: None
current value: None
a string
Pants version
$ pants --version
2.19.0
OS
MacOS on Apple silicon
pantsbuild/pantscool-easter-32542
04/19/2024, 4:11 PMpants check ::
in root dir, it does not work because it can't find pyproject.toml
This is different from pants' ruff integration where it is able to work with the pyproject.toml with no issues.
@bugzpodder ā /workspaces/example-python/helloworld (main) $ pyright
0 errors, 0 warnings, 0 informations
@bugzpodder ā /workspaces/example-python/helloworld (main) $ cd ..
@bugzpodder ā /workspaces/example-python (main) $ pants check
@bugzpodder ā /workspaces/example-python (main) $ pants check ::
16:05:24.63 [INFO] Completed: Force venv to materialize
16:05:29.04 [INFO] Completed: Run Pyright on 8 files.
16:05:29.04 [ERROR] Completed: Typecheck using Pyright - pyright - pyright failed (exit code 1).
helloworld/greet/greeting_test.py
helloworld/greet/greeting_test.py:9:26 - error: Argument of type "Literal['test']" cannot be assigned to parameter "name" of type "int" in function "greet"
"Literal['test']" is incompatible with "int" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations
ā pyright failed.
If you check the sandbox, you'll see it added pyrightconfig.json but it did not pick up pyproject.toml in helloworld
@bugzpodder ā /workspaces/example-python (main) $ ls /tmp/pants-sandbox-OCpc9c
__node helloworld requirements_venv.pex
__run.sh pyrightconfig.json requirements_venv.pex_bin_python_shim.sh
_binary_shims_d71438cc82ac7616b94db5246abc83b636b9bc4e66183a6ff021d42bf26ed483 requirements.pex requirements_venv.pex_pex_shim.sh
@bugzpodder ā /workspaces/example-python (main) $ ls /tmp/pants-sandbox-OCpc9c/helloworld/
__init__.py greet main.py translator
Pants version
2.20
OS
MacOS locally, the repo above is from a VSCode sandbox (linux)
Additional info
for running pyright outside of pants, it only recognizes the pyproject.toml in cwd (the place you run it in).
pantsbuild/pantscool-easter-32542
04/19/2024, 6:17 PMpants help-all
on my own repo I get:
$ pants help-all
11:11:51.53 [INFO] Initializing scheduler...
11:11:55.85 [INFO] Scheduler initialized.
11:11:56.19 [ERROR] type object 'MyCustomField' has no attribute 'help'
Use --print-stacktrace for more error details and/or -ldebug for more logs.
See <https://www.pantsbuild.org/v2.19/docs/troubleshooting> for common issues.
Consider reaching out for help: <https://www.pantsbuild.org/v2.19/docs/getting-help>
I would have expected either:
ā¢ help is a required parameter and pants help-all
fails with the above message.
ā¢ help is not a required parameter, and pants help-all
does not error.
Pants version
Which version of Pants are you using?
2.19.1
OS
Are you encountering the bug on MacOS, Linux, or both?
Linux
Additional info
Add any other information about the problem here, such as attachments or links to gists, if relevant.
Super minor issue, and I've fixed it by adding help text, but it seemed like the sort of inconsistency we should fix.
pantsbuild/pantscool-easter-32542
04/20/2024, 12:25 AMupdatedAt
values is a manual task, often overlooked by people updating the docs. This makes the docs appear more stale than they actually are (#18436).
With adhoc_tool
support, it should be possible to create a Pants-managed workflow to publish docs to readme.com.
We should be able to write a tool to update the front matter of the docs to have a correct updatedAt
value based on the mtime of the files, and then publish the post-processed files using the rdme
node.js-based tool.
pantsbuild/pantscool-easter-32542
04/21/2024, 7:34 PMcreate-react-app
or similar, what would they need to do to ensure everything they want works in Pants.
Or, if they're using typescript (which we don't support yet), how can they get their solution up and running.
This would be just single-backend recipes, not cross-backend stuff (yet)
pantsbuild/pantscool-easter-32542
04/22/2024, 4:11 PM--from
field of a docker COPY statement. Something like this:
docker_image(
name="deps",
...
)
docker_image(
...
instructions=[
...
"COPY --from=:deps /bin/app /bin/app",
...
],
)
where :deps
in --from=:deps
references the previous deps
stage.
Unfortunately, this doesn't seem to work right now.
Any tips / workarounds?
pantsbuild/pantscool-easter-32542
04/22/2024, 6:46 PM@rule
async def build_python_faas(
request: BuildPythonFaaSRequest,
) -> BuiltPackage:
additional_pex_args = (
# Ensure we can resolve manylinux wheels in addition to any AMI-specific wheels.
"--manylinux=manylinux2014",
# When we're executing Pex on Linux, allow a local interpreter to be resolved if
# available and matching the AMI platform.
"--resolve-local-platforms",
)
Looks like certain python libraries no longer release a linux version for manylinux2014. For example pyarrow:
| stderr:
build-1 | A distribution for pyarrow could not be resolved for cp312-cp312-linux_x86_64.
build-1 | Found 1 distribution for pyarrow that do not apply:
build-1 | 1.) The wheel tags for pyarrow 16.0.0 are cp312-cp312-manylinux_2_28_x86_64 which do not match the supported tags of cp312-cp312-linux_x86_64:
build-1 | cp312-cp312-manylinux2014_x86_64
build-1 | ... 122 more ...
There is no pyarrow x86 version after pyarrow 12.0.1.
I am using 2.21..0a0 pants version. This version seems to support python 312. I did change resolver_manylinux to manylinux_2_28 in pants.toml. This variable only applies for app.pex file does not impact the pex generation for FAAS.
Please let me know if you need more information.
pantsbuild/pantscool-easter-32542
04/22/2024, 10:04 PMcool-easter-32542
04/22/2024, 10:09 PMBuilding 34 requirements for requirements.pex from the 3rdparty/python/default.lock resolve: ...
My guess is that there is a dependency without a wheel matching our CI environment, but I'm having trouble figuring out 1) if that is the issue and 2) if so, which dependency it is. To start, I've been looking to produce more verbose output. (Nothing is emitted during the entire 15 minute build.) I tried PEX_VERBOSE=100 pants --level=debug --no-pantsd
, but that didn't quite do it. @lilatomic pointed me at this option, but that didn't do it either.
I can't tell if the issue is within pants or pex. I tried grepping for Buildint .* requirements for .*
in both repos, but didn't come up with anything promising.
Any suggestions? It would be nice if there was more output during that build process.
pantsbuild/pantscool-easter-32542
04/23/2024, 3:46 PMpants tailor
creates BUILD files that do not conform to this binary, so now users have to run pants tailor
and then manually go and run buildifier
.
Describe the solution you'd like
Two good solutions I can think of:
1. A tailor
option to support buildifier
style, or any other popular style.
2. A tailor
option to run an arbitrary program on the generated files.
Describe alternatives you've considered
We're living the alternative, which is to just do things manually. It could be better.
Additional context
None
pantsbuild/pantscool-easter-32542
04/23/2024, 4:53 PM~/.cache/pants/named_caches/pex_root
(<- Do your own homework here, I have not used Pants in ~1 year and location may have changed) or else you need to hop on the Pants dev release train to Pick up @huonw's patch asap.
pantsbuild/pantscool-easter-32542
04/25/2024, 6:49 AMpants dependencies --output-file=dependencies.json --format=json ...
should output dependencies to the specified file. It currently just prints to the console.
Pants version
v2.20
OS
both
Additional info
Looks like dependents
is also impacted by this bug. The fix looks simple enough. In list_dependencies_as_json
and list_dependents_as_json
we should replace
console.print_stdout(output)
with
with dependents_subsystem.line_oriented(console) as print_stdout:
print_stdout(output)
I can put together a PR for this.
pantsbuild/pants