Hi <@U04S8PLJWMT> i am running into this issue wit...
# replication-ask-ai
i
Hi @kapa.ai i am running into this issue with the postgres connector (both source & destination) Destination process exited with non-zero exit code 143
k
The knowledge sources don’t contain a Postgres‑specific root cause for exit code 143, but they do show that non‑zero exit code 143 is a generic Airbyte worker error that typically appears when a job is cancelled or the connector process is killed (often due to resource limits). You can see similar patterns in other connectors: • Source/destination processes exiting with code 143 are logged as “This warning is normal if the job was cancelled.” [Sync cancelled; Running sync worker cancellation] • In Kubernetes deployments, “Unable to find pod to retrieve exit value. Defaulting to value 143. This is expected if the job was cancelled.” appears for Postgres as well. [Postgres extraction issues] • Community replies often suggest increasing memory/CPU for the job when large data volumes are involved. [Custom source 143; Snowflake 143] Given that, for Postgres source and destination: 1. Check if the job is actually timing out or being cancelled ◦ Look in the sync logs for messages like “Running sync worker cancellation…”, “Job either timed out or was cancelled”, or “This warning is normal if the job was cancelled.” [Sync cancelled; Running sync worker cancellation] 2. Verify resource limits (especially on Kubernetes) ◦ Confirm
JOB_MAIN_CONTAINER_MEMORY_REQUEST
/
JOB_MAIN_CONTAINER_MEMORY_LIMIT
and CPU settings on scheduler/worker pods; try increasing memory (e.g. to 4–8 GiB) if you’re moving a lot of data or have complex normalization. [Custom source 143; Snowflake 143; Postgres extraction issues] 3. Test with a smaller/simpler sync ◦ For Postgres, try syncing only a subset of tables or a single stream to see if the error disappears; this has helped isolate problematic loads in similar cases. [Postgres extraction issues] 4. Check Postgres‑specific constraints ◦ For large volumes, remember Postgres is not a data warehouse and can struggle with big writes/typing‑deduping; monitor CPU/memory/IO on the DB during syncs. [Postgres troubleshooting] If you can share: • Deployment type (Docker vs Kubernetes), • Airbyte version, • Postgres source/destination versions, • Approximate data volume for the failing sync, then I can help you map these generic 143 patterns more precisely to your setup using the same references. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
i
@kapa.ai with smaller syncs it works.
k
Since smaller Postgres→Postgres syncs work and larger ones fail with exit code 143, this matches several reports where: • Jobs run fine on small volumes but fail or “get cancelled” once data size grows. [Postgres extraction issues; Custom source 143; Sync cancelled large data] • Exit code 143 is what Airbyte reports when the connector process is killed (often by resource limits or cancellation). [Sync cancelled; Running sync worker cancellation] From the knowledge sources, the concrete mitigations people used are: 1. Increase job resources (especially memory) ◦ Raise
JOB_MAIN_CONTAINER_MEMORY_REQUEST
and
JOB_MAIN_CONTAINER_MEMORY_LIMIT
on the worker/scheduler pods (Kubernetes) or equivalent container resources; community suggestions are to try 4–8 GiB when larger loads fail. [Custom source 143; Snowflake 143] ◦ Some users also doubled cluster resources when Postgres extractions were failing. [Postgres extraction issues] 2. Limit the scope of each sync ◦ Selecting only a subset of tables/streams allowed Postgres syncs that were failing at full scale to succeed. [Postgres extraction issues] ◦ Since you already see that “smaller syncs work”, you can formalize this by: ▪︎ Splitting tables into multiple connections, or ▪︎ Temporarily excluding the heaviest tables and adding them back one by one to find which ones trigger the failure. 3. Check for automatic cancellation / timeouts in logs ◦ Look for lines like “Running sync worker cancellation…”, “Job either timed out or was cancelled.” or “This warning is normal if the job was cancelled.” around the failure. [Sync cancelled; Running sync worker cancellation] ◦ If present, that confirms the worker is cancelling the job rather than a connector bug. 4. Upgrade Airbyte / connectors if you’re on an older version ◦ Several similar large‑volume issues were mitigated or investigated further after upgrading to newer Airbyte and connector versions. [Sync cancelled large data; Intermittent large Postgres→Snowflake] If you share your deployment type (Docker vs Kubernetes), Airbyte + connector versions, and approximate size of the failing sync, I can point to the closest matching case in the knowledge sources and the exact knobs they changed. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.