Akash
06/17/2022, 3:47 AMdius/pact-broker:2.100.0.1
) running on a k8s cluster reachable via an A record on a domain with a wildcard CA certificate. For some reason, our JVM provider verification is failing with the following error despite the docs mentioning that we shouldn’t need to do anything. Thoughts? Error:
au.com.dius.pact.core.pactbroker.InvalidNavigationRequest: Failed to fetch the root HAL document
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
JVM provider dependency:
testImplementation "au.com.dius.pact.provider:junit5spring:4.3.8"
A python consumer also fails to publish the pacts in CI because of a 400 error (I’m guessing it’s the same cause), though it works locally. Dependency: pact-python==1.5.2
.Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
400
for an invalid certificate. You wouldn’t get a valid HTTP response at allMatt (pactflow.io / pact-js / pact-go)
400
means bad request - a problem with the request.Akash
06/17/2022, 4:41 AMIssued by: Sectigo RSA Domain Validation Secure Server CA
I’m aware of a 400
being a bad request. Not sure why publishing the same pacts fails in CI but works locally against the same broker (the env variables are ...
publish_process = Popen(command)
publish_process.wait()
if publish_process.returncode != 0:
url = self._get_broker_base_url()
> raise RuntimeError(
f"There was an error while publishing to the pact broker at {url}.")
E RuntimeError: There was an error while publishing to the pact broker at <https://dev-broker.example.app>.
/root/.local/lib/python3.8/site-packages/pact/broker.py:94: RuntimeError
--------------------------- Captured stdout teardown ---------------------------
INFO going to shutdown ...
INFO WEBrick::HTTPServer#start done.
--------------------------- Captured stderr teardown ---------------------------
PactBroker::Client::Hal::ErrorResponseReturned - Error making request to <https://dev-broker.example.app> status=400
=========================== short test summary info ============================
ERROR tests/consumer/test_experiment_service_consumer.py::test_get_available_interventions
========================== 6 passed, 1 error in 1.95s ==========================
PactBroker::Client::Hal::ErrorResponseReturned - Error making request to <https://dev-broker.example.app> status=400
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/root/.local/lib/python3.8/site-packages/pact/pact.py", line 243, in stop_service
self.publish(
File "/root/.local/lib/python3.8/site-packages/pact/broker.py", line 94, in publish
raise RuntimeError(
RuntimeError: There was an error while publishing to the pact broker at <https://dev-broker.example.app>.
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Akash
06/17/2022, 4:46 AMMatt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Akash
06/17/2022, 4:49 AMMatt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Didn’t know there were certificates bundledFor clarification. We don’t bundle any Pactflow specific certs, but Java and other libs/browsers/tools often bundle well known certs together (e.g. https://curl.se/docs/caextract.html)
Akash
06/17/2022, 4:52 AMAkash
06/17/2022, 4:52 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/20/2022, 3:06 AMDEBUG=true
didn’t make a difference unfortunately.Akash
06/20/2022, 3:08 AMexport DEBUG=true # and other pact variables
ls tests/consumer/ | grep 'test_experiment' | \
xargs -n 1 -I '{}' python3 -m pytest -vv --log-level=DEBUG --log-cli-level=DEBUG tests/consumer/'{}'
Akash
06/20/2022, 4:10 AMDEBUG=true
?Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
--verbose
should do that I believeAkash
06/20/2022, 5:57 AMVERBOSE=true
flag. After setting that, I see this:
Reading environment variable exporting file contents.
Reading environment variable exporting file contents.
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Content-Type: text/plain\r\n"
-> "Content-Length: 0\r\n"
-> "Status: 400 Bad Request\r\n"
-> "Date: Mon, 20 Jun 2022 05:54:56 GMT\r\n"
-> "X-Powered-By: Phusion Passenger(R) 6.0.12\r\n"
-> "Server: nginx/1.18.0 + Phusion Passenger(R) 6.0.12\r\n"
-> "Via: 1.1 google\r\n"
-> "Alt-Svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n"
-> "\r\n"
reading 0 bytes...
-> ""
Despite the json files existing with correct read privileges..Akash
06/20/2022, 5:58 AMAkash
06/20/2022, 5:58 AMAkash
06/20/2022, 6:05 AM> docker run --rm pactfoundation/pact-cli:latest version
0.39.0
In CI, it’s 0.50.0
. Trying out 0.39.0.0
now in CI fingerscrossedAkash
06/20/2022, 6:06 AMMatt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Akash
06/20/2022, 6:29 AMExecuting command: export VERBOSE=true
------------------------------
Executing command: ls -lah myapp/src/tests/pacts
total 16K
drwxr-xr-x 2 root root 108 Jun 20 00:03 .
drwxr-xr-x 5 root root 132 Jun 20 01:54 ..
-rw-r--r-- 1 root root 3.5K Jun 20 06:02 myapp-provider1.json
-rw-r--r-- 1 root root 10.4K Jun 20 06:02 myapp-provider2.json
------------------------------
Executing command: /pact/entrypoint.sh version
0.50.0
------------------------------
Executing command: /pact/entrypoint.sh publish myapp/src/tests/pacts/myapp-provider1.json --consumer-app-version fake-git-sha-for-demo-1234567 --tag-with-git-branch
opening connection to <http://blah.example.app:443|blah.example.app:443>...
opened
starting SSL for <http://blah.example.app:443|blah.example.app:443>...
SSL established, protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384
<- "GET /? HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nAuthorization: [redacted]\r\n"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Content-Type: text/plain\r\n"
-> "Content-Length: 0\r\n"
-> "Status: 400 Bad Request\r\n"
-> "Date: Mon, 20 Jun 2022 06:02:37 GMT\r\n"
-> "X-Powered-By: Phusion Passenger(R) 6.0.12\r\n"
-> "Server: nginx/1.18.0 + Phusion Passenger(R) 6.0.12\r\n"
-> "Via: 1.1 google\r\n"
-> "Alt-Svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n"
-> "\r\n"
reading 0 bytes...
-> ""
read 0 bytes
Conn keep-alive
PactBroker::Client::Hal::ErrorResponseReturned - Error making request to <https://blah.example.app> status=400
Reading environment variable exporting file contents.
Reading environment variable exporting file contents.
Failed with exit code: 1
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Akash
06/20/2022, 6:42 AMAkash
06/20/2022, 7:24 AMTagged version fake-git-sha-for-demo-1234567 of myapp as "new/feature-branch"
Publishing myapp/provider1 pact to pact broker at <https://blah.example.app>
Regarding logs on broker, that was my first guess too, but I haven’t spotted any 🙂 The env variables we’re using on our deployment of image `dius/pact-broker:2.100.0.1`:
- name: PACT_BROKER_LOG_LEVEL
value: DEBUG
- name: PACT_BROKER_ALLOW_DANGEROUS_CONTRACT_MODIFICATION
value: 'false'
- name: PACT_BROKER_DATABASE_CONNECT_MAX_RETRIES
value: '10'
- name: PACT_BROKER_ALLOW_MISSING_MIGRATION_FILES
value: 'false'
- name: PACT_BROKER_DATABASE_MAX_CONNECTIONS
value: '1'
- name: PACT_BROKER_BASIC_AUTH_ENABLED
value: 'true'
- name: PACT_BROKER_PUBLIC_HEARTBEAT
value: 'true'
Also worth mentioning, I don’t think that specifying the SSL file/dir to the CLI tool has any effects. I tried specifying a SSL file and changed a character in it, and I didn’t receive any issues related to SSL connection establishment failure (this was using the local version 0.39.0
).Beth (pactflow.io/Pact Broker/pact-ruby)
VERBOSE
not DEBUG
, sorry for the bum steer.Beth (pactflow.io/Pact Broker/pact-ruby)
-> "\r\n"
reading 0 bytes...
-> ""
Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
$ curl -v <http://localhost:9292>
* Trying ::1:9292...
* Connected to localhost (::1) port 9292 (#0)
> GET / HTTP/1.1
> Host: localhost:9292
> User-Agent: curl/7.77.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Vary: Accept
< Content-Type: application/hal+json;charset=utf-8
< Content-Length: 4300
< Date: Wed, 22 Jun 2022 04:30:10 GMT
< Server: Webmachine-Ruby/1.6.0 Rack/1.3
< X-Pact-Broker-Version: 2.101.0
< X-Content-Type-Options: nosniff
< Connection: Keep-Alive
Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 3:05 AMpact-python==1.5.2
and hence, downgrading the pactfoundation/pact-cli:latest
image version to 0.39.0.0
. The problem is resolved since.
Regarding your query, we’re on version dius/pact-broker:2.100.0.1
and I curled both ports 9292 and 80. Results:
root@160b72d08493:/home/app/pact_broker# curl -v <http://localhost:9292>
* Trying 127.0.0.1:9292...
* TCP_NODELAY set
* connect to 127.0.0.1 port 9292 failed: Connection refused
* Trying ::1:9292...
* TCP_NODELAY set
* Immediate connect fail for ::1: Cannot assign requested address
* Trying ::1:9292...
* TCP_NODELAY set
* Immediate connect fail for ::1: Cannot assign requested address
* Failed to connect to localhost port 9292: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 9292: Connection refused
root@160b72d08493:/home/app/pact_broker#
root@160b72d08493:/home/app/pact_broker#
root@160b72d08493:/home/app/pact_broker# curl -v <http://localhost:80>
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Content-Type: text/plain
< Content-Length: 0
< Connection: keep-alive
< Status: 401 Unauthorized
< WWW-Authenticate: Basic realm="Restricted area"
< Date: Thu, 23 Jun 2022 03:03:30 GMT
< X-Powered-By: Phusion Passenger(R) 6.0.12
< Server: nginx/1.18.0 + Phusion Passenger(R) 6.0.12
<
* Connection #0 to host localhost left intact
root@160b72d08493:/home/app/pact_broker#
Matt (pactflow.io / pact-js / pact-go)
Akash
06/23/2022, 3:15 AMroot@160b72d08493:/home/app/pact_broker# curl -v <http://localhost:9292> -u "user2:password"
* Trying 127.0.0.1:9292...
* TCP_NODELAY set
* connect to 127.0.0.1 port 9292 failed: Connection refused
* Trying ::1:9292...
* TCP_NODELAY set
* Immediate connect fail for ::1: Cannot assign requested address
* Trying ::1:9292...
* TCP_NODELAY set
* Immediate connect fail for ::1: Cannot assign requested address
* Failed to connect to localhost port 9292: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 9292: Connection refused
root@160b72d08493:/home/app/pact_broker#
root@160b72d08493:/home/app/pact_broker# curl -v <http://localhost:80> -u "user2:password"
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
* Server auth using Basic with user 'user2'
> GET / HTTP/1.1
> Host: localhost
> Authorization: Basic dXNlcjI6cGFzc3dvcmQ=
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/hal+json;charset=utf-8
< Content-Length: 4160
< Connection: keep-alive
< Status: 200 OK
< Date: Thu, 23 Jun 2022 03:13:10 GMT
< Vary: Accept
< X-Content-Type-Options: nosniff
< Server: Webmachine-Ruby/1.6.0 Rack/1.3
< X-Pact-Broker-Version: 2.100.0
< X-Powered-By: Phusion Passenger(R) 6.0.12
<
{"_links":{"self":{"href":"<http://localhost>","title":"Index","templated":false},"pb:publish-pact":{"href":"<http://localhost/pacts/provider/{provider}/consumer/{consumer}/version/{consumerApplicationVersion}>","title":"Publish a pact","templated":true},"pb:publish-contracts":{"href":"<http://localhost/contracts/publish>","title":"Publish contracts","templated":false},"pb:latest-pact-versions":{"href":"<http://localhost/pacts/latest>","title":"Latest pact versions","templated":false},"pb:tagged-pact-versions":{"href":"<http://localhost/pacts/provider/{provider}/consumer/{consumer}/tag/{tag}>","title":"All versions of a pact for a given consumer, provider and consumer version tag","templated":false},"pb:pacticipants":{"href":"<http://localhost/pacticipants>","title":"Pacticipants","templated":false},"pb:pacticipant":{"href":"<http://localhost/pacticipants/{pacticipant}>","title":"Fetch pacticipant by name","templated":true},"pb:latest-provider-pacts":{"href":"<http://localhost/pacts/provider/{provider}/latest>","title":"Latest pacts by provider","templated":true},"pb:latest-provider-pacts-with-tag":{"href":"<http://localhost/pacts/provider/{provider}/latest/{tag}>","title":"Latest pacts for provider with the specified tag","templated":true},"pb:provider-pacts-with-tag":{"href":"<http://localhost/pacts/provider/{provider}/tag/{tag}>","title":"All pact versions for the provider with the specified consumer version tag","templated":true},"pb:provider-pacts":{"href":"<http://localhost/pacts/provider/{provider}>","title":"All pact versions for the specified provider","templated":true},"pb:latest-version":{"href":"<http://localhost/pacticipants/{pacticipant}/latest-version>","title":"Latest pacticipant version","templated":true},"pb:latest-tagged-version":{"href":"<http://localhost/pacticipants/{pacticipant}/latest-version/{tag}>","title":"Latest pacticipant version with the specified tag","templated":true},"pb:webhooks":{"href":"<http://localhost/webhooks>","title":"Webhooks","templated":false},"pb:webhook":{"href":"<http://localhost/webhooks/{uuid}>","title":"Webhook","templated":true},"pb:integrations":{"href":"<http://localhost/integrations>","title":"Integrations","templated":false},"pb:pacticipant-version-tag":{"href":"<http://localhost/pacticipants/{pacticipant}/versions/{version}/tags/{tag}>","title":"Get, create or delete a tag for a pacticipant version","templated":true},"pb:pacticipant-branch-version":{"href":"<http://localhost/pacticipants/{pacticipant}/branches/{branch}/versions/{version}>","title":"Get or add/create a pacticipant version for a branch","templated":true},"pb:pacticipant-version":{"href":"<http://localhost/pacticipants/{pacticipant}/versions/{version}>","title":"Get, create or delete a pacticipant version","templated":true},"pb:metrics":{"href":"<http://localhost/metrics>","title":"Get Pact Broker metrics"},"pb:can-i-deploy-pacticipant-version-to-tag":{"href":"<http://localhost/can-i-deploy?pacticipant={pacticipant}&version={version}&to={tag}>","title":"Determine if an application version can be safely deployed to an environment identified by the given tag","templated":true},"pb:can-i-deploy-pacticipant-version-to-environment":{"href":"<http://localhost/can-i-deploy?pacticipant={pacticipant}&version={version}&environment={environment}>","title":"Determine if an application version can be safely deployed to an environment","templated":true},"pb:provider-pacts-for-verification":{"href":"<http://localhost/pacts/provider/{provider}/for-verification>","title":"Pact versions to be verified for the specified provider","templated":true},"beta:provider-pacts-for-verification":{"name":"beta","href":"<http://localhost/pacts/provider/{provider}/for-verification>","title":"DEPRECATED - please use pb:provider-pacts-for-verification","templated":true},"curies":[{"name":"pb","href":"<http://localhost/doc/{rel}?context=index>","templated":true},{"name":"beta","href":"<http://localhost/doc/{rel}?context=index>","templated":true}],"pb:environments":{"title":"Environments","href":"<http://localhost/environments>","templated":false},"pb:environment":{"title":"Environment","* Connection #0 to host localhost left intact
root@160b72d08493:/home/app/pact_broker#
Beth (pactflow.io/Pact Broker/pact-ruby)
< Server: Webmachine-Ruby/1.6.0 Rack/1.3
< X-Pact-Broker-Version: 2.100.0
Beth (pactflow.io/Pact Broker/pact-ruby)
0.39.0.0
is over a year oldAkash
06/23/2022, 3:18 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
0.50.0.29
tag of the pact docker cli?Akash
06/23/2022, 3:47 AMlatest
was previously pointing to in CI, since we don’t have access to the docker daemon. Here are my results from a quick few tests:Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 3:48 AMAkash
06/23/2022, 3:48 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 3:50 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 3:52 AM0.39.0
(dunno which patch version) when publishing was failing for the Python and pact-cli (0.59.0
) on CIAkash
06/23/2022, 3:54 AMpact-python==1.5.2
locally to k8s dev too.Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 3:56 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
<- "GET /? HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nAuthorization: [redacted]\r\n"
Akash
06/23/2022, 4:06 AM--tag-with-git-branch
anymore.Beth (pactflow.io/Pact Broker/pact-ruby)
Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 4:08 AMBeth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/23/2022, 4:10 AMExecuting command: export VERBOSE=true
------------------------------
Executing command: ls -lah myapp/src/tests/pacts
total 16K
drwxr-xr-x 2 root root 108 Jun 23 01:39 .
drwxr-xr-x 5 root root 132 Jun 23 01:39 ..
-rw-r--r-- 1 root root 3.5K Jun 23 03:24 myapp-provider1.json
-rw-r--r-- 1 root root 10.4K Jun 23 03:24 myapp-provider2.json
------------------------------
Executing command: /pact/entrypoint.sh version
0.39.0
------------------------------
Executing command: /pact/entrypoint.sh publish myapp/src/tests/pacts --consumer-app-version $PACT_CONSUMER_VERSION
Publishing myapp/provider1 pact to pact broker at <https://blah.example.app>
The latest version of this pact can be accessed at the following URL:
<https://blah.example.app/pacts/provider/provider1/consumer/myapp/latest>
Publishing myapp/provider2 pact to pact broker at <https://blah.example.app>
The latest version of this pact can be accessed at the following URL:
<https://blah.example.app/pacts/provider/provider2/consumer/myapp/latest>
Successfully ran freestyle step: Publish Python pacts
Reading environment variable exporting file contents.
Reading environment variable exporting file contents.
Beth (pactflow.io/Pact Broker/pact-ruby)
Akash
06/24/2022, 3:18 AMMatt (pactflow.io / pact-js / pact-go)
Akash
06/24/2022, 7:15 AMBeth (pactflow.io/Pact Broker/pact-ruby)