https://pact.io logo
Join Slack
Powered by
Hi - I'm using pact-broker `2.96.0.0` and have hit...
# pact-broker
a
Hi - I'm using pact-broker 
2.96.0.0
and have hit an error trying to delete an integration between 2 pacticipants via the broker UI:
Copy code
Could not delete resources due to error: An error has occurred. The details have been logged with the reference QkphWxELnm

Error reference: QkphWxELnm
I'm looking in the pact broker logs for some matching reference but can't see anything. I've not had a problem deleting integrations before. Any clues as to how to investigate further would be most welcome.
m
hmm strange, it should be in the logs. There are no errors in your Pact Broker logs at all?
a
I'm running pb in k8s with log level set to debug, but after a flurry of logging (sql, properties dump) on startup there's hardly anything in the logs at all. Feels like I'm missing something in setup.
🤔 1
m
mind sharing a redacted startup log for us?
a
Will do - AFK right now but will do so in the next hour.
👍 1
I guess I could also uprev to latest pb version, providing its not going to drop any data.
m
It shouldn't. But also the logs should be there unless you happen to be very unluckily on a version that had some bug (I don't recall but it's possible)
a
These are logs - almost all of the activity is from startup and issuing further requests from the pact broker UI don't seem to trigger any further logging activity.
pb.log.txt
And this is the configuration setup in the YAML::
Copy code
spec:
      containers:
      - env:
        - name: DATASOURCE_HOST
          value: ****
        - name: DATASOURCE_PORT
          value: "5432"
        - name: DATASOURCE_USERNAME
          valueFrom:
            secretKeyRef:
              key: postgres_username
              name: postgres-secrets
        - name: DATASOURCE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: postgres_password
              name: postgres-secrets
        - name: PACT_BROKER_DATABASE_URL
          value: postgres://$(DATASOURCE_USERNAME):$(DATASOURCE_PASSWORD)@$(DATASOURCE_HOST):$(DATASOURCE_PORT)/pact-broker
        - name: PACT_BROKER_WEBHOOK_HOST_WHITELIST
          value: ***
        - name: PACT_BROKER_DISABLE_SSL_VERIFICATION
          value: "true"
        - name: PACT_BROKER_LOG_LEVEL
          value: DEBUG
        image: dius/pact-broker:2.96.0.0
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 9m
            memory: 1372Mi
m
You’re right, definitely an older version. So strange there are no logs after the initial boot. There should be a lot more, IIUC
An upgrade is worth doing, provided two things: 1. You have an up-to-date snapshot of the DB 2. You have a way to resurrect it and deploy the previous broker back on top of it
(it should be safe, the database migrations are pretty well tested)
a
OK I'll backup the existing DB and go for 
2.110.0
💪 1
OK I discovered (indirectly) that this was probably a sql db timeout; upping 
PACT_BROKER_DATABASE_STATEMENT_TIMEOUT
from the default (15) to 120 removed the error; seems like whatever an integration deletion is doing, when the tables get big enough it can struggle.
👍 1
... I think that if the integration is the last one in which a given pacticipant appears, the broker also removes the pacticipant. That may explain some of the extra effort going on behind the scenes.
m
ah, that makes some form of sense - the cascade delete could be quite big if there are a lot of rows
a
Is there a safe way to prune some of the faster-growing tables? I've noticed that 
webhook_executions
and 
triggered_webhooks
are getting pretty big, and I'm not sure of the utility of keeping an ever-growing record of those things. But I would not be so foolhardy as to just remove rows at the SQL level. Also having the ability to prune undeployed versions from 
deployed_versions
in bulk would be good. And without knowing the inner workings of the pb, the issues with SQL timeouts when deleting a "busy" pacticipant might be due to a combination of ever-growing tables and a lack of suitable SQL indexes, but that's just speculation on my part.
Open in Slack
PreviousNext
Powered by