Hey Team! I am new to Airbyte . I came across as e...
# replication-troubleshooting
a
Hey Team! I am new to Airbyte . I came across as error while running my airbyte source i.e. extracting data from my custom source and syncing it to PostgreSQL destination.My connection is getting cancelled without my input.Logs are:-
2022-08-15 10:33:37 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):325 - Records read: 225000 (203 MB) 2022-08-15 10:42:39 destination > 2022-08-15 10:42:39 INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(lambda$flushAll$1):86 - Flushing <stream_name>: 7435 records (24 MB)
2022-08-15 10:52:16 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):191 - Running sync worker cancellation...
2022-08-15 10:52:16 INFO i.a.w.g.DefaultReplicationWorker(cancel):444 - Cancelling replication worker...
2022-08-15 10:52:26 INFO i.a.w.g.DefaultReplicationWorker(cancel):452 - Cancelling destination...
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteDestination(cancel):125 - Attempting to cancel destination process...
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteDestination(cancel):130 - Destination process exists, cancelling...
2022-08-15 10:52:26 INFO i.a.w.g.DefaultReplicationWorker(run):175 - One of source or destination thread complete. Waiting on the other.
2022-08-15 10:52:26 WARN i.a.c.i.LineGobbler(voidCall):86 - airbyte-destination gobbler IOException: Stream closed. Typically happens when cancelling a job.
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteDestination(cancel):132 - Cancelled destination process!
2022-08-15 10:52:26 INFO i.a.w.g.DefaultReplicationWorker(cancel):459 - Cancelling source...
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteSource(cancel):142 - Attempting to cancel source process...
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteSource(cancel):147 - Source process exists, cancelling...
2022-08-15 10:52:26 INFO i.a.w.g.DefaultReplicationWorker(run):177 - Source and destination threads complete.
2022-08-15 10:52:26 INFO i.a.w.i.DefaultAirbyteSource(cancel):149 - Cancelled source process!
2022-08-15 10:52:26 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):195 - Interrupting worker thread...
2022-08-15 10:52:26 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):198 - Cancelling completable future...
2022-08-15 10:52:26 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
2022-08-15 10:52:26 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
What could be the issue here? I am assuming it is because my data is more than 200MB.If this is how could I resolve it?
e
Job either timed out or was cancelled.
It`s prob timeout - try pulling less data to verify this if so - change the code in the source to have larger timeout
a
I tried with less data it was successful there.To change the timeout do I have to change the airbyte/.env --
MAX_SYNC_TIMEOUT_DAYS=3
1