Subject: Server OOM with 16 connections and large catalogs on 2.1.0 We're running Airbyte 2.1.0 OSS on EKS. The airbyte-server pod OOMs every ~1 hour with Java heap space at -Xmx8g. Setup: - 16 connections, one with 688 streams (IBM i source with plan explosion) - Server has 4 HikariCP pools + 4 more (8 total in 2.1) - io-executor thread pool grows to 95+ threads per cycle - Every connection triggers applySchemaChange with "Diff is empty" — no actual changes - Upgraded from 2.0.1 where the ConnectionRead.toString() bug was dumping full catalogs into logs — that's fixed in 2.1, but OOM persists Questions: 1. Is there a way to limit the io-executor-thread pool size on the server? (e.g., IO_TASK_EXECUTOR_THREADS or Micronaut config) 2. Can we disable schema propagation discovery for specific connections via API/config without changing the UI setting? 3. Is the server expected to hold all connection catalogs in memory simultaneously, or is this a known leak? 4. Any recommended -Xmx for 16 connections with one 688-stream catalog? 5. Why does 2.1 create 8 HikariCP pools vs 4 in 2.0.1? Logs showing the pattern: applySchemaChange → Diff is empty → Sending notification → (repeat 16x) → OOM

The knowledge sources provided don't contain enough specific information to fully answer your detailed questions about airbyte-server internals in 2.1.0. However, here's what I can address based on the available documentation: What the Knowledge Sources Cover OOM handling in 2.1.0: Airbyte 2.1 added the

-XX:+ExitOnOutOfMemoryError

flag to all connector JVMs so they exit cleanly instead of hanging on OOM, and a new

connector_exit_code

metric is emitted on non-zero exits. [Airbyte 2.1 Release Notes] However, this applies to connector JVMs, not the airbyte-server pod itself. JDBC buffer memory (connector-side): There is an open PR reducing the JDBC

TARGET_BUFFER_SIZE_RATIO

from 60% to 10% of JVM heap, with a 500MB hard cap, specifically to address OOM kills on large catalogs (400+ streams). [JDBC fetchSize PR] This is a CDK-level fix affecting JDBC source connectors, not the server pod. Async schema discovery in 2.1: Schema discovery now runs asynchronously to prevent UI blocking on large schemas. [Airbyte 2.1 Release Notes] Schema change propagation: You can configure per-connection schema change behavior (propagate all, propagate fields only, manual approval, or stop syncs) via the connection Settings tab. [Schema Change Management] This is UI-based; the knowledge sources don't document an API method to disable schema discovery per-connection without touching the UI setting. General memory scaling: The docs recommend starting with mid-sized instances and tuning, and note that the Source worker is the main memory culprit during syncs. [Scaling Airbyte] What I Cannot Answer The knowledge sources do not contain information about: 1.

IO_TASK_EXECUTOR_THREADS

or Micronaut-level thread pool configuration for the server 2. API-based disabling of schema propagation discovery per-connection 3. Whether the server holds all catalogs in memory simultaneously or if this is a known leak 4. Recommended

-Xmx

sizing for 16 connections with a 688-stream catalog 5. Why 2.1 creates 8 HikariCP pools vs 4 in 2.0.1 For these specific questions, I'd recommend opening a GitHub issue or checking the Airbyte GitHub discussions, as these appear to be server-internals questions that go beyond what's documented in the available sources. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.