Hi Airbyte, We are seeing failed syncs for a large snowflake source, I am attaching the full log file. Is there anything in these logs that indicate the cause? It looks like this may be the relevant error:

2022-11-08 19:42:36 [32mINFO[m i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):381 - Total records read: 16928693 (31 GB)
2022-11-08 19:42:36 [32mINFO[m i.a.w.g.DefaultReplicationWorker(cancel):501 - Cancelling destination...
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):121 - Attempting to cancel destination process...
2022-11-08 19:42:36 [32mINFO[m i.a.w.g.DefaultReplicationWorker(run):190 - One of source or destination thread complete. Waiting on the other.
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):126 - Destination process exists, cancelling...
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(destroy):664 - (pod: homelander-airbyte-external / destination-postgres-write-30-0-cekpy) - Destroying Kube process.
2022-11-08 19:42:36 [32mINFO[m i.a.w.g.DefaultReplicationWorker(run):192 - Source and destination threads complete.
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(close):737 - (pod: homelander-airbyte-external / destination-postgres-write-30-0-cekpy) - Closed all resources for pod
2022-11-08 19:42:36 [33mWARN[m i.a.c.i.LineGobbler(voidCall):119 - airbyte-destination gobbler IOException: Socket closed. Typically happens when cancelling a job.
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(destroy):670 - (pod: homelander-airbyte-external / destination-postgres-write-30-0-cekpy) - Destroyed Kube process.
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteDestination(cancel):128 - Cancelled destination process!
2022-11-08 19:42:36 [32mINFO[m i.a.w.g.DefaultReplicationWorker(cancel):508 - Cancelling source...
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):141 - Attempting to cancel source process...
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):146 - Source process exists, cancelling...
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(destroy):664 - (pod: homelander-airbyte-external / source-snowflake-read-30-0-dbrwk) - Destroying Kube process.
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(close):737 - (pod: homelander-airbyte-external / source-snowflake-read-30-0-dbrwk) - Closed all resources for pod
2022-11-08 19:42:36 [33mWARN[m i.a.c.i.LineGobbler(voidCall):119 - airbyte-source gobbler IOException: Socket closed. Typically happens when cancelling a job.
2022-11-08 19:42:36 [32mINFO[m i.a.w.p.KubePodProcess(destroy):670 - (pod: homelander-airbyte-external / source-snowflake-read-30-0-dbrwk) - Destroyed Kube process.
2022-11-08 19:42:36 [32mINFO[m i.a.w.i.DefaultAirbyteSource(cancel):148 - Cancelled source process!
2022-11-08 19:42:36 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$5):230 - Interrupting worker thread...
2022-11-08 19:42:36 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$5):233 - Cancelling completable future...
2022-11-08 19:42:36 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
2022-11-08 19:42:36 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
2022-11-08 19:42:36 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):162 - Stopping cancellation check scheduling...
2022-11-08 19:42:36 [1;31mERROR[m i.a.w.WorkerUtils(gentleClose):53 - Exception while while waiting for process to finish
022-11-08 19:42:36 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
2022-11-08 19:42:36 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):162 - Stopping cancellation check scheduling...
2022-11-08 19:42:36 [1;31mERROR[m i.a.w.WorkerUtils(gentleClose):53 - Exception while while waiting for process to finish
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep0(Native Method) ~[?:?]
	at java.lang.Thread.sleep(Thread.java:465) ~[?:?]
	at java.lang.Process.waitFor(Process.java:468) ~[?:?]
	at io.airbyte.workers.process.KubePodProcess.waitFor(KubePodProcess.java:653) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.WorkerUtils.gentleClose(WorkerUtils.java:50) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:128) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:194) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:71) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$4(TemporalAttemptExecution.java:190) ~[io.airbyte-airbyte-workers-0.40.17.jar:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2022-11-08 19:42:36 [32mINFO[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):283 - Stopping temporal heartbeating...
2022-11-08 19:42:36 [1;31mERROR[m i.a.w.g.DefaultReplicationWorker(run):196 - Sync worker failed.
io.airbyte.workers.exception.WorkerException: Source process exit with code 143. This warning is normal if the job was cancelled.
	at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:135) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:194) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:71) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$4(TemporalAttemptExecution.java:190) ~[io.airbyte-airbyte-workers-0.40.17.jar:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
	Suppressed: io.airbyte.workers.exception.WorkerException: Destination process exit with code 143. This warning is normal if the job was cancelled.
		at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:115) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:151) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:71) ~[io.airbyte-airbyte-commons-worker-0.40.17.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$4(TemporalAttemptExecution.java:190) ~[io.airbyte-airbyte-workers-0.40.17.jar:?]
		at java.lang.Thread.run(Thread.java:1589) ~[?:?]

Hey Coleman, there have been a lot of reported issues with snowflake recently. I’ll take a look at the logs and will follow up with any questions

Thanks very much for your reply @Saj Dider (Airbyte). One thing to note is that smaller syncs from snowflake seem to work, it is the larger ones that fail. If it is possible to determine whether the issue is: • communication with the source • resources on the airbyte pods • or communication with the destination Then we can experiment with increasing resources in the relevant area.

Hello Coleman Kelleghan, it's been a while without an update from us. Are you still having problems or did you find a solution?

Hi @Marcos Marx (Airbyte), we have not resolved the issues syncing large snowflake sources

Hey @Coleman Kelleghan sorry for the long delay, could you try upgrading the snowflake connector and then posting the sync logs here?