Hi one more issue we were using the Pact broker docker image Pact Foundation #pact-broker

Hi, one more issue - we were using the Pact broke...

Abhi Nandan

03/29/2022, 3:26 AM

Hi, one more issue - we were using the Pact broker docker image version

2.84.0.0

, and we are currently trying to upgrade it to the latest version

2.96.0.0

. I changed the docker-compose file with the version and tried to spin up the pact broker and it throws error as it is unable to connect to the database instance, below is the error -

Copy code

pact-broker_1  | /pact_broker/vendor/bundle/ruby/2.7.0/gems/anyway_config-2.3.0/lib/anyway/type_casting.rb:111: warning: Pattern matching is experimental, and the behavior may change in future versions of Ruby!
pact-broker_1  | /pact_broker/vendor/bundle/ruby/2.7.0/gems/anyway_config-2.3.0/lib/anyway/rbs.rb:40: warning: Pattern matching is experimental, and the behavior may change in future versions of Ruby!
pact-broker_1  | Puma starting in single mode...
pact-broker_1  | * Puma version: 5.6.2 (ruby 2.7.5-p203) ("Birdie's Version")
pact-broker_1  | *  Min threads: 0
pact-broker_1  | *  Max threads: 5
pact-broker_1  | *  Environment: production
pact-broker_1  | *          PID: 7
pact-broker_1  | 2022-03-29 03:08:07.303198 I [7:4640] pact-broker -- Connecting to database: -- "{:adapter=>\"postgres\", :user=>\"<username>\", :password=>\"*****\", :host=>\"xxxx.xxxx.xxxx\", :database=>\"pact_broker\", :port=>6432, :encoding=>\"utf8\", :sql_log_level=>:debug, :enable_caller_logging=>false, :log_warn_duration=>5.0, :pool_timeout=>5, :driver_options=>{:options=>\"-c statement_timeout=15s\"}, :connect_max_retries=>5}"
pact-broker_1  | 2022-03-29 03:08:07.457332 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 5 tries to go.
pact-broker_1  | 2022-03-29 03:08:10.572607 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 4 tries to go.
pact-broker_1  | 2022-03-29 03:08:13.688064 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 3 tries to go.
pact-broker_1  | 2022-03-29 03:08:16.803917 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 2 tries to go.
pact-broker_1  | 2022-03-29 03:08:19.921639 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 1 tries to go.
pact-broker_1  | ! Unable to load application: Sequel::DatabaseConnectionError: PG::ConnectionBad: FATAL:  unsupported startup parameter: options
pact-broker_1  | bundler: failed to load command: puma (/pact_broker/vendor/bundle/ruby/2.7.0/bin/puma)
pact-broker_1  | Sequel::DatabaseConnectionError: PG::ConnectionBad: FATAL:  unsupported startup parameter: options

while reverting it to

2.84.0.0

starts the pact broker normally. Is there any known compatibility issues with upgrading?

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:28 AM

I haven't seen that one before

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:28 AM

I'll try and repro

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:32 AM

I can't reproduce that issue in the docker compose I'm using.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:32 AM

Can you share yours?

Abhi Nandan

03/29/2022, 3:40 AM

Copy code

version: "3"

services:
  pact-broker:
    image: pactfoundation/pact-broker:2.84.0.0
    ports:
      - "9292:9292"
    environment:
      PACT_BROKER_PORT: '9292'
      PACT_BROKER_DATABASE_URL: "xxxxxxxx"
      PACT_BROKER_LOG_LEVEL: INFO
      PACT_BROKER_SQL_LOG_LEVEL: DEBUG
      # PACT_BROKER_DATABASE_CONNECT_MAX_RETRIES is only needed for docker-compose
      # because the database takes longer to start up than the puma process
      # Should not be needed in production.
      PACT_BROKER_DATABASE_CONNECT_MAX_RETRIES: "5"
      # The list of allowed base URLs (not setting this makes the app vulnerable to cache poisoning)
      # This allows the app to be addressed from the host from within another docker container correctly
      PACT_BROKER_BASE_URL: 'xxxxxxxx'

  # Nginx is not necessary, but demonstrates how
  # one might use a reverse proxy in front of the broker,
  # and includes the use of a self-signed TLS certificate
  nginx:
    image: nginx:alpine
    depends_on:
      - pact-broker
    volumes:
      - ./ssl/nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - ./ssl:/etc/nginx/ssl
    ports:
      - "443:443"
      - "80:80"
volumes:
  postgres-volume:

this is my

docker-compose.yml

, masked the

PACT_BROKER_DATABASE_URL

and

PACT_BROKER_BASE_URL

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:40 AM

I probably can't reproduce it if you have an external database

Abhi Nandan

03/29/2022, 3:41 AM

yes, we have an external database, and 2.84.0.0 connects to it successfully

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:41 AM

what version of postgres is it?

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:42 AM

also, do you have a stack trace?

Abhi Nandan

03/29/2022, 3:42 AM

stack trace of?

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:42 AM

the error

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:42 AM

Sequel::DatabaseConnectionError: PG::ConnectionBad: FATAL:  unsupported startup parameter: options

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:43 AM

I tried to reproduce the issue locally by not starting up the postgres database, and I got a big long stracktrace.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:44 AM

Copy code

pact-broker_1  | ! Unable to load application: Sequel::DatabaseConnectionError: PG::ConnectionBad: could not translate host name "postgres" to address: Name does not resolve
pact-broker_1  | bundler: failed to load command: puma (/pact_broker/vendor/bundle/ruby/2.7.0/bin/puma)
pact-broker_1  | Sequel::DatabaseConnectionError: PG::ConnectionBad: could not translate host name "postgres" to address: Name does not resolve
pact-broker_1  |
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pg-1.3.4/lib/pg/connection.rb:702:in `connect_start'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pg-1.3.4/lib/pg/connection.rb:702:in `new'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/adapters/postgres.rb:208:in `connect'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool.rb:122:in `make_new'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool/threaded.rb:209:in `assign_connection'
...

Abhi Nandan

03/29/2022, 3:45 AM

Copy code

pact-broker_1  | /pact_broker/vendor/bundle/ruby/2.7.0/gems/anyway_config-2.3.0/lib/anyway/type_casting.rb:111: warning: Pattern matching is experimental, and the behavior may change in future versions of Ruby!
pact-broker_1  | /pact_broker/vendor/bundle/ruby/2.7.0/gems/anyway_config-2.3.0/lib/anyway/rbs.rb:40: warning: Pattern matching is experimental, and the behavior may change in future versions of Ruby!
pact-broker_1  | Puma starting in single mode...
pact-broker_1  | * Puma version: 5.6.2 (ruby 2.7.5-p203) ("Birdie's Version")
pact-broker_1  | *  Min threads: 0
pact-broker_1  | *  Max threads: 5
pact-broker_1  | *  Environment: production
pact-broker_1  | *          PID: 7
pact-broker_1  | 2022-03-29 03:08:07.303198 I [7:4640] pact-broker -- Connecting to database: -- "{:adapter=>\"postgres\", :user=>\"<username>\", :password=>\"*****\", :host=>\"xxxx.xxxx.xxxx\", :database=>\"pact_broker\", :port=>6432, :encoding=>\"utf8\", :sql_log_level=>:debug, :enable_caller_logging=>false, :log_warn_duration=>5.0, :pool_timeout=>5, :driver_options=>{:options=>\"-c statement_timeout=15s\"}, :connect_max_retries=>5}"
pact-broker_1  | 2022-03-29 03:08:07.457332 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 5 tries to go.
pact-broker_1  | 2022-03-29 03:08:10.572607 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 4 tries to go.
pact-broker_1  | 2022-03-29 03:08:13.688064 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 3 tries to go.
pact-broker_1  | 2022-03-29 03:08:16.803917 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 2 tries to go.
pact-broker_1  | 2022-03-29 03:08:19.921639 I [7:4640] pact-broker -- Error connecting to database (Sequel::DatabaseConnectionError). Waiting 3 seconds and trying again. 1 tries to go.
pact-broker_1  | ! Unable to load application: Sequel::DatabaseConnectionError: PG::ConnectionBad: FATAL:  unsupported startup parameter: options
pact-broker_1  | bundler: failed to load command: puma (/pact_broker/vendor/bundle/ruby/2.7.0/bin/puma)
pact-broker_1  | Sequel::DatabaseConnectionError: PG::ConnectionBad: FATAL:  unsupported startup parameter: options
pact-broker_1  | 
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pg-1.3.4/lib/pg/connection.rb:637:in `async_connect_or_reset'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pg-1.3.4/lib/pg/connection.rb:707:in `new'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/adapters/postgres.rb:208:in `connect'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool.rb:122:in `make_new'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool/threaded.rb:209:in `assign_connection'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool/threaded.rb:139:in `acquire'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/connection_pool/threaded.rb:91:in `hold'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/database/connecting.rb:269:in `synchronize'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/database/connecting.rb:278:in `test_connection'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/database/misc.rb:175:in `initialize'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/database/connecting.rb:57:in `new'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/database/connecting.rb:57:in `connect'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/sequel-5.54.0/lib/sequel/core.rb:124:in `connect'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/initializers/database_connection.rb:17:in `block in create_database_connection'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/initializers/database_connection.rb:31:in `with_retries'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/initializers/database_connection.rb:16:in `create_database_connection'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/app.rb:135:in `configure_database_connection'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/app.rb:99:in `post_configure'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/pact_broker-2.96.0/lib/pact_broker/app.rb:50:in `initialize'
pact-broker_1  |   <http://config.ru:4:in|config.ru:4:in> `new'
pact-broker_1  |   <http://config.ru:4:in|config.ru:4:in> `block in <main>'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:116:in `eval'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:116:in `new_from_string'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:105:in `load_file'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:66:in `parse_file'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/configuration.rb:348:in `load_rackup'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/configuration.rb:270:in `app'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/runner.rb:150:in `load_and_bind'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/single.rb:44:in `run'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/launcher.rb:182:in `run'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/lib/puma/cli.rb:81:in `run'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/gems/puma-5.6.2/bin/puma:10:in `<top (required)>'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/bin/puma:23:in `load'
pact-broker_1  |   /pact_broker/vendor/bundle/ruby/2.7.0/bin/puma:23:in `<top (required)>'
pact-broker_1 exited with code 1

Abhi Nandan

03/29/2022, 3:46 AM

postgres version we are using is 13.5

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:48 AM

should be fine.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:48 AM

Just looking where the error is being raised from

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:48 AM

https://github.com/ged/ruby-pg/blob/v1.3.4/lib/pg/connection.rb#L637

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 3:49 AM

can you try running postgres 13.5 in your docker compose and see what happens?

Abhi Nandan

03/29/2022, 3:52 AM

I will check and let you know

Abhi Nandan

03/29/2022, 4:22 AM

running with a fresh instance of postgres 13.5 in

docker-compose

spins up the pact broker successfully -

Copy code

postgres:
    image: postgres:13.5-bullseye
    healthcheck:
      test: psql postgres --command "select 1" -U postgres
    volumes:
      - postgres-volume:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: xxxxxxx
      POSTGRES_PASSWORD: xxxxxxxx
      POSTGRES_DB: postgres

Abhi Nandan

03/29/2022, 5:02 AM

@Beth (pactflow.io/Pact Broker/pact-ruby) I have investigated further and seems to be some change in

2.87.1

is causing this issue, I am able to upgrade till

2.87.0

, but when I try with

2.87.1

, it starts throwing this error.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:04 AM

in docker compose, or only against your database?

Abhi Nandan

03/29/2022, 5:04 AM

only against my database, it's a hosted instance of postgres 13.5

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:05 AM

so, it seems like there's something slightly different about connecting to your external instance vs the local postgres instance

Abhi Nandan

03/29/2022, 5:05 AM

with

postgres:13.5-bullseye

image, I am able to start even the latest version

2.96.0.0

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:06 AM

So, I'm taking a wild guess at a fix, based on the error

unsupported startup parameter: options

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:06 AM

see the options listed in here:

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:06 AM

Copy code

"{:adapter=>\"postgres\", :user=>\"<username>\", :password=>\"*****\", :host=>\"xxxx.xxxx.xxxx\", :database=>\"pact_broker\", :port=>6432, :encoding=>\"utf8\", :sql_log_level=>:debug, :enable_caller_logging=>false, :log_warn_duration=>5.0, :pool_timeout=>5, :driver_options=>{:options=>\"-c statement_timeout=15s\"}, :connect_max_retries=>5}"

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:06 AM

{:options=>\"-c statement_timeout=15s\"}

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:07 AM

perhaps for some reason, that options statement doesn't work for your external database.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:07 AM

I'm just releasing a new version of the broker that will allow you to set that statement timeout to nil, and when it is nil, it won't set any options.

🙏 1

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:08 AM

that may fix it. it may not. We can but try.

Abhi Nandan

03/29/2022, 5:08 AM

thanks a lot Beth

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:09 AM

it's just building now. when it comes out, try setting

PACT_BROKER_DATABASE_STATEMENT_TIMEOUT: "nil"

in your docker compose.

👍 1

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:09 AM

then have a look at the logs and you should see that there is no

{:options=>\"-c statement_timeout=15s\"}

in there.

Abhi Nandan

03/29/2022, 5:10 AM

sure

Abhi Nandan

03/29/2022, 5:22 AM

it's fixed, thanks a lot for your support, really appreciate it! 🙏

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:23 AM

you're welcome. I don't understand why it didn't work, but good that it's fixed.

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:23 AM

So, the risk is, if you get really long SQL queries, your queries won't time out (with the statement timeout set to nil)

Abhi Nandan

03/29/2022, 5:24 AM

got it, meanwhile I will also check why the options are not being supported by the DB instance that we are using

Beth (pactflow.io/Pact Broker/pact-ruby)

03/29/2022, 5:24 AM

👍🏼

3 Views

Open in Slack

Previous Next