For OPAL server restart issues with SIGTERM and WebSocket disconnections, here are debugging steps: Check Network/DNS Configuration When the OPAL client cannot maintain a stable connection to the server, you might see repeated connection attempts in the logs (1) :

2021-06-27T15:58:12.564384+0300 |fastapi_websocket_pubsub.pub_sub_client | INFO | Trying to connect to Pub/Sub server - <ws://localhost:7002/ws>  
2021-06-27T15:58:12.564687+0300 |fastapi_websocket_rpc.websocket_rpc_client | INFO | Trying server - <ws://localhost:7002/ws>  
2021-06-27T15:58:12.567187+0300 |fastapi_websocket_rpc.websocket_rpc_client | INFO | RPC connection was refused by server  
...

This indicates networking or DNS issues in your setup . Increase Log Verbosity To get more detailed logs for debugging, configure these environment variables : •

OPAL_LOG_MODULE_EXCLUDE_LIST=[]

- To get all logs and ensure nothing is filtered •

OPAL_LOG_LEVEL

- Set to

DEBUG

to get more logs (default is

INFO

) •

OPAL_INLINE_OPA_LOG_FORMAT

- Set to

http

for nicer formatting if running OPA inline Production Configuration For production deployments, ensure you're using the proper backbone pub/sub configuration (2) :

OPAL_BROADCAST_URI=<postgres://localhost/mydb> opal-server run --engine-type gunicorn

The backbone pub/sub (Kafka, Postgres LISTEN/NOTIFY, or Redis) is required for all OPAL-servers to stay synced . Resource Issues Based on a community discussion, OPAL servers can experience memory issues under high load. With ~100 RPS to Kafka and 150 client pods, OPAL server pods with 4 CPU and 4Gi RAM limits experienced OOM crashes after about 22 minutes (3) . The issue showed increasing CPU usage hitting limits, followed by RAM usage increase leading to crashes . To help our engineers investigate furt

To learn more, see the sources I considered: [[1] Troubleshooting](https://docs.opal.ac/getting-started/running-opal/troubleshooting) [[2] Running the OPAL Server and Client in Production](https://docs.opal.ac/getting-started/running-opal/as-python-package/running-in-prod) [[3] How to properly scale out OPAL servers for data updates directly from Kafka](https://github.com/permitio/opal/discussions/671)

Figured out the issue. The server was not able to reach the database. Dns resolution failure or incorrect password throw verbose error logs. This was a networking issue where the port was not reachable.

There were no logs to identify this.