https://linen.dev logo
Join Slack
Powered by
<@U04S8PLJWMT> On airbyte 1.8.5, salesforce source...
# replication-ask-ai
k
@kapa.ai On airbyte 1.8.5, salesforce source and postgres destination, near the end after ingestion is done following 
WITH deduped_source AS
query we got a connection leak exception.
Copy code
2026-08-27 18:43:19 destination WARN HikariPool-1:housekeeper c.z.h.p.ProxyLeakTask(run):85 Connection leak detection triggered for org.postgresql.jdbc.PgConnection@18907fb3 on thread DefaultDispatcher-worker-3, stack trace follows java.lang.Exception: Apparent connection leak detected
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:99) ~[HikariCP-7.0.2.jar:?]
	at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.execute(PostgresAirbyteClient.kt:455) ~[io.airbyte.airbyte-integrations.connectors-destination-postgres.jar:?]
	at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.upsertTable-Fn9JJlk(PostgresAirbyteClient.kt:171) ~[io.airbyte.airbyte-integrations.connectors-destination-postgres.jar:?]
	at io.airbyte.cdk.load.table.directload.DirectLoadTableDedupStreamLoader.teardown(DirectLoadTableStreamLoader.kt:129) ~[bulk-cdk-core-load-1.0.20.jar:?]
	at io.airbyte.cdk.load.dataflow.DestinationLifecycle$finalizeIndividualStreams$2$1$1.invokeSuspend(DestinationLifecycle.kt:96) ~[bulk-cdk-core-load-1.0.20.jar:?]
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-2.1.10.jar:2.1.10-release-473]
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:?]
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:?]
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:811) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:?]
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:715) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:?]
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:702) ~[kotlinx-coroutines-core-jvm-1.8.1.jar:?]

Stack Trace: java.lang.Exception: Apparent connection leak detected
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:99)
	at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.execute(PostgresAirbyteClient.kt:455)
	at io.airbyte.integrations.destination.postgres.client.PostgresAirbyteClient.upsertTable-Fn9JJlk(PostgresAirbyteClient.kt:171)
	at io.airbyte.cdk.load.table.directload.DirectLoadTableDedupStreamLoader.teardown(DirectLoadTableStreamLoader.kt:129)
	at io.airbyte.cdk.load.dataflow.DestinationLifecycle$finalizeIndividualStreams$2$1$1.invokeSuspend(DestinationLifecycle.kt:96)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:811)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:715)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:702)
What could we even do about that?
k
HikariCP Connection Leak Warning During Upsert This is a known pattern with the destination-postgres connector. Here's what's happening and what you can do: What's Happening The warning 
Apparent connection leak detected
from HikariCP does not necessarily mean an actual leak occurred. HikariCP fires this when a single connection is held longer than the 
leakDetectionThreshold
, which in the postgres destination is set to 
DATA_SOURCE_CONNECTION_TIMEOUT_MS + 10000L
. [PostgresBeanFactory] The 
WITH deduped_source AS ...
upsert CTE is a long-running maintenance query (dedup finalization). On large Salesforce tables, it can exceed the threshold, triggering the warning while the connection is still legitimately in use inside 
PostgresAirbyteClient.execute
. [PostgresAirbyteClient] Related Known Issues 1. 
statement_timeout
cancellations — If your Postgres server or connection pooler (e.g., Supabase's pooler) enforces a low 
statement_timeout
, the long-running upsert can be actively cancelled with 
ERROR: canceling statement due to statement timeout
. PR #82251 fixes this (in connector version 3.0.14+) by setting 
SET SESSION statement_timeout = 0
on each pooled connection. [fix PR] 2. `COUNT()` stalls on large tables* — There's an open issue (#82229) where a full 
COUNT(*)
at sync start causes multi-minute stalls on large tables, which can compound with the upsert timing. [COUNT issue] What You Can Do 1. Check if the sync actually failed or just warned. If the sync completed successfully, the leak detection warning is cosmetic — the connection was returned to the pool after the query finished. 2. Check for active 
statement_timeout
on your Postgres server. If the upsert is being cancelled (not just slow), you'll see a 
PSQLException: canceling statement due to statement timeout
error. You can override this via 
jdbcUrlParams
in the destination config:options=-c statement_timeout=0Note: connector version 3.0.14+ handles this automatically unless you set it explicitly. [fix PR] 3. Tune PostgreSQL settings for long-running queries, similar to what resolved a large-table issue in another case:work_mem = 72MB client_connection_check_interval = 30000 tcp_keepalives_idle = 10[GitHub issue #51580] 4. Upgrade the destination-postgres connector to 3.0.14 or later if possible, as the 
statement_timeout = 0
fix is included there. [fix PR] The key question is whether the sync is failing (upsert cancelled) or just slow (leak warning only). The stack trace you shared is a WARN, not an ERROR, so the sync may have completed — worth checking the final sync status. 👍 0 button 👎 0 button ℹī¸ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements
2 Views
Open in Slack
PreviousNext
Powered by