Hi all, I'm experiencing memory issues on our Airbyte 2.0.1 instance deployed by abctl 0.33. This is on RHEL 8.10 on an AWS EC2 instance with 8 cpus and 64GB of ram. Does anyone have any insight on what I should be doing to get around this error?

DefaultDispatcher-worker-3 o.a.p.h.InternalParquetRecordWriter(flushRowGroupToStore):203 Too much memory used: Store {
 
2026-02-23 11:08:05 replication-orchestrator ERROR SourceReader error:
2026-02-23 11:08:05 replication-orchestrator ERROR runJobs failed; recording failure but continuing to finish.
2026-02-23 11:08:05 replication-orchestrator ERROR Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteDestination@7c6edbd2; recording failure but continuing.
2026-02-23 11:08:06 source ERROR java.io.IOException: Broken pipe
2026-02-23 11:08:06 source ERROR 	at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
2026-02-23 11:08:06 source ERROR 	at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
2026-02-23 11:08:06 source ERROR 	at java.base/sun.nio.ch.SocketChannelImpl.tryWrite(SocketChannelImpl.java:1371)
2026-02-23 11:08:06 source ERROR 	at java.base/sun.nio.ch.SocketChannelImpl.blockingWriteFully(SocketChannelImpl.java:1405)
2026-02-23 11:08:06 source ERROR 	at java.base/sun.nio.ch.SocketOutputStream.write(SocketOutputStream.java:58)
2026-02-23 11:08:06 source ERROR 	at java.base/java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:170)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.output.sockets.SocketProtobufOutputConsumer.withLockFlush(SocketProtobufOutputConsumer.kt:69)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.output.sockets.SocketProtobufOutputConsumer.close(SocketProtobufOutputConsumer.kt:77)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.read.FeedBootstrap$ProtoEfficientStreamRecordConsumer.close(FeedBootstrap.kt:227)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.output.OutputMessageRouter.close(OutputMessageRouter.kt:154)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.read.JdbcPartitionReader.releaseResources(JdbcPartitionReader.kt:83)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.read.FeedReader.readPartitionWithResources(FeedReader.kt:255)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.read.FeedReader.access$readPartitionWithResources(FeedReader.kt:35)
2026-02-23 11:08:06 source ERROR 	at io.airbyte.cdk.read.FeedReader$asyncReadPartition$readerJob$1.invokeSuspend(FeedReader.kt:193)
2026-02-23 11:08:06 source ERROR 	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:811)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:715)
2026-02-23 11:08:06 source ERROR 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:702)
2026-02-23 11:08:06 source ERROR 	Suppressed: java.io.IOException: Broken pipe
2026-02-23 11:08:06 source ERROR 		... 22 more
2026-02-23 11:09:05 replication-orchestrator WARN Failed to wait for exit value file /source/exitCode.txt to be found.
2026-02-23 11:09:05 replication-orchestrator ERROR Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource@78db6b8e; recording failure but continuing.

This happened when I deployed with the following values.yaml file 'abctl local install --values values.yaml', and below is the values I have set

worker:
  job:
    resources:
      destination:
        limits:
          memory: 8Gi   # or 10Gi/12Gi if needed
          cpu: 2
        requests:
          memory: 4Gi
          cpu: 1

Hi @Phil Pierce, today I also noticed the destination container wasn't picking up global configs from my values.yaml. My solution was brute force update the connection in the airbyte db, which is honored first. Airbyte db in postgres:

select * from public.connection;

--Update Connection resources
update airbyte.public.connection
set resource_requirements = jsonb_build_object(
  'cpu_request', '1',
  'cpu_limit',   '2',
  'memory_request','4Gi',
  'memory_limit',  '8Gi'
)
where id = 'yourconnectionid';

@Karen Connelly Thank you, this fixed our issue! We have been trying to chance this down for almost 2 weeks at this point.

Oh wow! I'm so glad to help! I'm actually a newbie and was just encountering the exact same problem setting up our new env!