https://linen.dev logo
Join Slack
Powered by
Hi team, I submitted an issue a month ago, and the...
# help-connector-development
h
Hi team, I submitted an issue a month ago, and the AI autoreply said they would have the issue routed to oncall, but I have no clue how to track the oncall progress. This one is fairly important for us github.com/airbytehq/airbyte/issues/80250 any ideas?
github.com/airbytehq/airbyte/issues/80250 Hi all want to follow up is there any way to find status of oncall issues?
I have submitted 3 issues, each have been triaged via the AI triage and some have had PRs created and reviewed and approved! I have been stuck for a month on the same bug. How can this happen? How is this possible? How can I track oncall status? Is there an oncall that looks at bugs? Am I just wasting my time hoping airbyte will fix things or should I move to a different ETL?
Airbyte has now failed out my main connection with non-descriptive errors. Idk how to resolve this without a multi-day full refresh. How can I ensure that anybody looks at the PRs and issues myself and my team contributes get eyes on this so we don't need to go through this randomly?
šŸ‘€ 1
Copy code
2026-08-02 12:15:56 replication-orchestrator INFO Failures: [ {
  "failureOrigin" : "source",
  "internalMessage" : "Source process exited with non-zero exit code 2",
  "externalMessage" : "Something went wrong within the source connector",
  "metadata" : {
    "attemptNumber" : 0,
    "jobId" : 9682,
    "connector_command" : "read"
  },
  "stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 2\n\tat io.airbyte.container.orchestrator.worker.SourceReader.run(ReplicationTask.kt:210)\n\tat io.airbyte.container.orchestrator.worker.SourceReader$run$1.invokeSuspend(ReplicationTask.kt)\n\tat kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)\n\tat kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n",
  "timestamp" : 1785694556837
} ]
2026-08-02 12:15:56 replication-orchestrator INFO
2026-08-02 12:15:56 replication-orchestrator INFO Closing io.micronaut.context.propagation.instrument.execution.ContextPropagatingExecutorService@2445cad
2026-08-02 12:15:56 replication-orchestrator INFO ----- END REPLICATION -----
2026-08-02 12:15:56 replication-orchestrator INFO
2026-08-02 12:15:56 replication-orchestrator INFO Closing io.micronaut.context.propagation.instrument.execution.ContextPropagatingExecutorService@1f22d7e4
2026-08-02 12:15:56 replication-orchestrator INFO Closing io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteDestination@311f8821
2026-08-02 12:15:57 replication-orchestrator INFO Closing io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource@2a37e02b
2026-08-02 12:15:57 replication-orchestrator ERROR Error closing resource io.airbyte.container.orchestrator.worker.io.LocalContainerAirbyteSource@2a37e02b; recording failure but continuing.
2026-08-02 12:15:57 replication-orchestrator INFO Closing io.airbyte.container.orchestrator.worker.RecordSchemaValidator@61cc39f8
2026-08-02 12:15:57 replication-orchestrator INFO Closing io.airbyte.container.orchestrator.persistence.SyncPersistenceImpl@fbba04a
2026-08-02 12:15:57 replication-orchestrator INFO Returning output...
2026-08-02 12:15:57 platform INFO
Copy code
2026-08-02 12:13:50 source WARN main i.a.c.i.b.IntegrationRunner$Companion(stopOrphanedThreads):469                  The main thread is exiting while children non-daemon threads from a connector are still active.
                 Ideally, this situation should not happen...
                 Please check with maintainers if the connector or library code should safely clean up its threads before quitting instead.
                 The main thread is: main (RUNNABLE)
Thread stacktrace: java.base/java.lang.Thread.getStackTrace(Thread.java:2451)
       at io.airbyte.cdk.integrations.base.IntegrationRunner$Companion.dumpThread(IntegrationRunner.kt:525)
       at io.airbyte.cdk.integrations.base.IntegrationRunner$Companion.access$dumpThread(IntegrationRunner.kt:366)
       at io.airbyte.cdk.integrations.base.IntegrationRunner$Companion$stopOrphanedThreads$1.invoke(IntegrationRunner.kt:474)
       at io.github.oshai.kotlinlogging.internal.MessageInvokerKt.toStringSafe(MessageInvoker.kt:5)
       at io.github.oshai.kotlinlogging.slf4j.internal.LocationAwareKLogger.warn$lambda$7(LocationAwareKLogger.kt:191)
       at <http://io.github.oshai.kotlinlogging.slf4j.internal.LocationAwareKLogger.at|io.github.oshai.kotlinlogging.slf4j.internal.LocationAwareKLogger.at>(LocationAwareKLogger.kt:43)
       at io.github.oshai.kotlinlogging.slf4j.internal.LocationAwareKLogger.warn(LocationAwareKLogger.kt:191)
       at io.airbyte.cdk.integrations.base.IntegrationRunner$Companion.stopOrphanedThreads(IntegrationRunner.kt:469)
       at io.airbyte.cdk.integrations.base.IntegrationRunner$Companion.stopOrphanedThreads$default(IntegrationRunner.kt:458)
       at io.airbyte.cdk.integrations.base.IntegrationRunner.readSerial(IntegrationRunner.kt:294)
       at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.kt:190)
       at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.kt:119)
       at io.airbyte.cdk.integrations.base.IntegrationRunner.run$default(IntegrationRunner.kt:113)
       at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.kt)
       at io.airbyte.integrations.source.mongodb.MongoDbSource.main(MongoDbSource.java:54)
2026-08-02 12:13:50 source WARN main i.a.c.i.b.IntegrationRunner$Companion(stopOrphanedThreads):489 Active non-daemon thread info: debezium-mongodbconnector-mongodb_srv___miter-prod.ptjem.mongodb.net__readPreference_secondary-replicator-fetcher-0 (RUNNABLE)
 Thread stacktrace: java.base/sun.nio.ch.Net.poll(Native Method)
        at java.base/sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
        at java.base/sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:201)
        at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:309)
        at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
        at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
        at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
        at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
        at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
        at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
        at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1461)
        at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1066)
        at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:175)
        at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:200)
        at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:739)
        at com.mongodb.internal.connection.InternalStreamConnection.receiveMessageWithAdditionalTimeout(InternalStreamConnection.java:603)
        at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:451)
        at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:372)
        at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:114)
        at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:765)
        at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:76)
        at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:209)
        at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:115)
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:83)
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:74)
        at com.mongodb.internal.connection.DefaultServer$OperationCountTrackingConnection.command(DefaultServer.java:299)
        at com.mongodb.internal.operation.QueryBatchCursor.lambda$getMore$1(QueryBatchCursor.java:285)
        at com.mongodb.internal.operation.QueryBatchCursor$ResourceManager.executeWithConnection(QueryBatchCursor.java:514)
        at com.mongodb.internal.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:282)
        at com.mongodb.internal.operation.QueryBatchCursor.tryHasNext(QueryBatchCursor.java:235)
        at com.mongodb.internal.operation.QueryBatchCursor.lambda$tryNext$0(QueryBatchCursor.java:218)
        at com.mongodb.internal.operation.QueryBatchCursor$ResourceManager.execute(QueryBatchCursor.java:407)
        at com.mongodb.internal.operation.QueryBatchCursor.tryNext(QueryBatchCursor.java:217)
        at com.mongodb.internal.operation.ChangeStreamBatchCursor.lambda$tryNext$4(ChangeStreamBatchCursor.java:97)
        at com.mongodb.internal.operation.ChangeStreamBatchCursor.resumeableOperation(ChangeStreamBatchCursor.java:189)
        at com.mongodb.internal.operation.ChangeStreamBatchCursor.tryNext(ChangeStreamBatchCursor.java:95)
        at com.mongodb.client.internal.MongoChangeStreamCursorImpl.tryNext(MongoChangeStreamCursorImpl.java:90)
        at io.debezium.connector.mongodb.events.BufferingChangeStreamCursor$EventFetcher.fetchEvent(BufferingChangeStreamCursor.java:259)
        at io.debezium.connector.mongodb.events.BufferingChangeStreamCursor$EventFetcher.fetchEvents(BufferingChangeStreamCursor.java:246)
        at io.debezium.connector.mongodb.events.BufferingChangeStreamCursor$EventFetcher.run(BufferingChangeStreamCursor.java:225)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
2026-08-02 12:13:50 source INFO main i.a.c.i.b.IntegrationRunner(runInternal):224 Completed integration: io.airbyte.integrations.source.mongodb.MongoDbSource
2026-08-02 12:13:50 source INFO main i.a.i.s.m.MongoDbSource(main):55 completed source: class io.airbyte.integrations.source.mongodb.MongoDbSource
2026-08-02 12:15:50 source ERROR pool-3-thread-1 i.a.c.i.b.IntegrationRunner$Companion(stopOrphanedThreads$lambda$5):507 Failed to interrupt children non-daemon threads, forcefully exiting NOW...
2026-08-02 12:15:50 replication-orchestrator ERROR Source process exited with non-zero exit code 2
2026-08-02 12:15:50 replication-orchestrator ERROR SourceReader error:
r
Chatting internally about this.
šŸ™ 1
šŸ”„ 1
I see that you're talking to Rodi about this on the issue. Please let me know if you need anything else
h
thanks @Richelle Herrli appreciate the attention here
octavia loves 1
Hey @Richelle Herrli havent gotten any correspondence in ~5 days on this now šŸ˜ž
r
Hi Henry, it looks like Rodi offered to cut a modified connector for you so that you could test whether it's an issue of timeouts being too short. Did you take him up on that?
h
I didn't explicitly. He asked for more information and I provided to him
r
Got it. What would you like moving forward? Do you want to try extending the timeout period? Or are you looking for an update on a longer term fix?
h
Tbh was looking for a few things: ā€¢ github.com/airbytehq/airbyte/pull/80161 was created from our original issue github.com/airbytehq/airbyte/issues/80160 ā€¢ github.com/airbytehq/airbyte/pull/80213 was created from issue github.com/airbytehq/airbyte/issues/80211#ā€¦ ā€¢ github.com/airbytehq/airbyte/issues/80250 (issue that was forwarded to oncall) was created 3rd here. I wanted to give Rodi the full context on the problems we were facing with mongo before he spent time creating a modified connector. Ideally someone from the team could evaluate if those two PRs would be helpful for solving the problems we are facing and then merge them if so and we would use the new connector version instead. Would that work
loading 1
r
Update: We're noodling on it. The issues you're bringing us have complicated moving parts, so we're going to figure out the best approach to solve them.
h
appreciate it!!!
Hey @Richelle Herrli checking in here. Having some strange problems that are very similar to what are described in the tickets above. These ones are thankfully recovered but for about 12 hours we had 101 records load every hour, then the connector returned to normal. We have moved our biggest collection off airbyte to mitigate the un-recoverable veresions of this problem but our customers are asking questions about data freshness. Can we help at all?
Hey @Richelle Herrli checking in again. Has the team gotten a chance to look at this at all?
šŸ‘€ 1
r
Still in progress, as far as I know. I'm checking in again
šŸ™ 1
h
Hi @Richelle Herrli saw someone started working on this and stopped: github.com/airbytehq/airbyte/pull/80213
We had the 101 records synced thing happen again and connector has not synced successfully since yesterday morning. https://airbytehq.slack.com/archives/C01AHCD885S/p1792596290850749
https://airbytehq.slack.com/archives/C01AHCD885S/p1792597012654249?thread_ts=1792596290.850749&amp;cid=C01AHCD885S pretty funny that now the kapa.ai is now referring me to open issues on how to solve this that I filed
r
Hi Henry! TL;DR: The PR doesn't address the root cause of the issue. Our engineers are working on another PR to address the issue.
šŸ™Œ 1
h
okay! great to hear
@Richelle Herrli just saw the issues resolved with a mongo v3 upgrade. Just want to say thank you so much for putting up with me šŸ™‚ and big thanks to you and your team for driving this through!!!!
šŸ™ 1
r
Hi Henry, I was just in a meeting where we talked about how happy you would be to hear about the MongoDB updates, haha. Thanks for sticking with us and let me know if you have any other issues šŸ˜Š
2 Views
Open in Slack
PreviousNext
Powered by