https://pact.io logo
Join Slack
Powered by
Have a brief investigation (using Claude) on suppo...
# maintainers
r
Have a brief investigation (using Claude) on supporting Python for plugins. Proposal written up in github.com/pact-foundation/pact-plugins/blob/…/010_Support_Python_script_plugins.md
👀 1
@Joshua Ellis @Matt (pactflow.io / pact-js / pact-go) could you have a read through it, it is not as clean as what Lua support is, but will not be as bad as what JavaScript would be
j
I feel like the 'no system deps' constraint is too severe when it comes to Python, and will cause too many issues. Lua is great because it has such a small runtime footprint, and people like Python because it has a huge stdlib and packages to do basically anything. Trying to prevent people from accessing the full stdlib or wheels is going to be difficult. Even if we do tell plugin authors 'don't use XYZ from stdlib', the fact that the proposal contemplates supporting wheels means that we have to tell authors they can use XYZ from the script OR any of the dependencies. I think it may be worth considering that Lua and Python fill different niches: • Lua: for simpler plugins that don't need access to much else. Guaranteed to run everywhere. • Python: for more complex plugins that might required stdlib/packages from the broader ecosystem. Requires 
libpython
to be present, so should run most places, but not everywhere Python has a stable ABI, which we presumably can leverage. This means that as long as the user has a relatively recent version of Python on their system, the plugin should run just fine.
r
That is a typical response from a Python person 😛
😆 1
j
Well, you asked for my opinion 😛
r
Just imagine a user who is not a Python person, but a Java Windows user. They find out there is a plugin that supports what they need. They add it using the DSL, and their test then fails to run. They check the test output, and find out they now need to get their IT to install Python
I would call that a terrible user experience
j
I guess it depends on what you want the Python plugin to achieve. In my view, there's no point supporting Python as a scripting language unless you can guarantee support of its stdlib and broader ecosystem. You might as well just use Lua and not Python.
Hence why I said that, in my view, Python and Lua would fill different niches
I also don't see why the Windows user couldn't just download 
Python.dll
and place it in the plugins directory so that the Pact can link against it?
r
Supporting the stdlib and ecosystem is not the issue, I don't have a problem with that (there may be some complexity in having to resolve system architectures to get the correct wheels). What I have an issue is forcing users to have to install additional software outside of what they are using. Plugins should be encapsulated, and I feel that should be a hard constraint.
If we make it a requirement to have to have an external additional runtime, we may as well drop Python all together and just support Node.
I'm also happy to only support Lua, it is a great language and embeds really well, but as @Matt (pactflow.io / pact-js / pact-go) mentioned we got lots of push back from users who don't want to have to learn it.
That may not be an issue for plugin authors, they have the choice now of using Lua for a simpler implementation or a gRPC executable plugin in in their language of choice
BTW, the Avro plugin is written in scala and requires a JVM to run. I think that has not been an issue as it has probably not been used by anyone who is not already using a JVM
j
So then, is it or isn't it a hard constraint to have (or not have) an external dependency requirement on plugins?
r
It is, but the problem with the executable model we have no control over the plugin author's implementation and can't do anything other than say Bad Plugin Author!
ok, I'm thinking of shelving this for now
a
Hi, I'm the author of the github.com/ZirekHQ/pact-avro-plugin I have been thinking of migrating the plugin to Rust because I'm trying to improve my Rust, currently next to none. That being said
The Reality of Building 
pact-avro-plugin
in Scala (JVM) Choosing Scala for the Avro plugin was a pragmatic architectural decision driven by three main realities: 1. Definitive Avro Semantics (
org.apache.avro
) The reference Apache Avro Java implementation has always been the golden standard. Complex schema evolution, datum readers/writers, logical types (e.g., timestamps, decimals), and Confluent Schema Registry wire formats are fully supported and battle-tested on the JVM. Trying to replicate exact Avro evolution edge cases in non-JVM ports often leads to subtle deserialization discrepancies. 2. Expressive Functional Modeling Pact content matching requires parsing ASTs, comparing payload diffs, extracting matching rules, and recursively walking schema hierarchies. Scala’s pattern matching, case classes, and type-level safety made modeling complex Avro schema ASTs and diff trees straightforward and robust. 3. Akka/Pekko gRPC & JVM Tooling Integrating standard gRPC code generation (
ScalaPB
/ 
fs2-grpc
/ Pekko gRPC) allowed strong type safety across the protobuf contracts required by the Pact plugin driver. The Friction Points Encountered on the JVM While Scala/JVM solved the Avro schema compatibility problem effortlessly, it introduced specific distribution and lifecycle headaches: • Startup & Process Spawning Overhead: Pact plugins are spawned as child processes on demand. A fat JAR requires a JVM startup cycle (plus requiring users to have Java 17+ pre-installed in their test runner environments). • Cross-Language Consumer Friction: A non-JVM developer (e.g., Python, Go, TypeScript) using the Avro plugin shouldn't ideally need a full JRE installed just to run their local contract tests. • Packaging Complexity: Solving the distribution problem via GraalVM Native Image to emit static binaries requires configuring reflection configs for Avro's runtime reflection and gRPC netty transports. The Verdict: Stick with Scala or Rewrite in Rust? DimensionKeep / Evolve in Scala (JVM)Port to RustAvro Correctness10/10 — Native Apache Avro engine; zero risk with custom logical types or Schema Registry wire formats.7/10 — 
apache-avro
crate is solid, but lacks complete feature parity with Java Avro edge cases.*Plugin Distribution*Requires Java 17+ runtime on user machines or GraalVM Native Image build complexity.10/10 — Pure static binary per OS/architecture; zero runtime prereqs for users.*Pact Alignment*Second-party JVM driver integration.10/10 — Direct alignment with Pact core Rust crates (
pact_models
, 
pact_plugin_driver
).*Maintenance Cost*Low if you already maintain the codebase and understand the Scala ecosystem.High upfront rewrite effort to replicate the full matching engine logic. Architectural Conclusion • If maintaining feature depth and enterprise Avro parity is priority #1: Stay with Scala. The reliability of the official Java Avro library outweighs the inconvenience of a Java prerequisite, particularly since the vast majority of enterprise teams relying heavily on Kafka + Avro already run JVM tooling in their build pipelines. • If frictionless, universal zero-prerequisite distribution across all Pact ecosystems is the goal: Port the core matching logic to Rust, using 
tonic
for gRPC and 
apache-avro
, wrapping it as a multi-arch static binary release.
r
Hello @Ali Ustek, thanks for the summary. And that is one of the main contentions, how to make it easier for users without making it extremely difficult for framework/plugin authors. Interestingly, for gRPC/Protobuf we had the opposite experience. The rust protobuf crates were very well written and easy to use, while the Java and Go ones carried a lot of older baggage. An example, to add inject-able authentication, with rust is was trivial to add an interceptor, but with Pact-JVM is was fairly convoluted to do the same thing. I think the reason is that the rust crates are newer and could just focus on the current specs.
BTW, I haven't actually heard of anyone complaining about having to install a JVM to use the avro plugin. I think if users need it they will just install it
✅ 1
3 Views
Open in Slack
PreviousNext
Powered by