*Is this your first time deploying Airbyte*: Yes
*...
# troubleshootingj
Jean Sahlberg
03/24/2022, 12:53 PMIs this your first time deploying Airbyte: Yes
OS Version / Instance: MacOS Monterey (M1)
Memory / Disk: 16Gb / 1Tb
Deployment: Docker
Airbyte Version: 0.35.59-alpha
Source name/version: Zendesk Support 0.2.2
Destination name/version: Redshift/CSV/JSON (tried all)
Step: On sync
Description: When extracting data from Zendesk “tickets” the worker just stops, tried multiple times, even using an Intel Mac, have the same problem… It always stops in the same part [Records read: 54000 (208 MB)]. I’ve tested other tables from Zendesk (organizations, users, etc…) and they apparently are working fine. I was able to full extract the tickets table using an old version of Airbyte (0.29.11).
Jean Sahlberg
03/24/2022, 12:53 PM2022-03-24 11:53:49 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):301 - Records read: 50000 (193 MB)2022-03-24 11:54:07 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):301 - Records read: 51000 (197 MB)2022-03-24 11:54:14 destination > 2022-03-24 11:54:14 INFO i.a.i.d.b.BufferedStreamConsumer(flushQueueToDestination):181 - Flushing buffer: 26201189 bytes2022-03-24 11:54:14 destination > 2022-03-24 11:54:14 INFO i.a.i.d.b.BufferedStreamConsumer(lambda$flushQueueToDestination$1):185 - Flushing zendesk_tickets: 1705 records2022-03-24 11:54:14 destination > 2022-03-24 11:54:14 INFO i.a.i.d.r.RedshiftSqlOperations(insertRecordsInternal):43 - actual size of batch: 17052022-03-24 11:54:25 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):301 - Records read: 52000 (200 MB)2022-03-24 11:54:43 destination > 2022-03-24 11:54:43 INFO i.a.i.d.b.BufferedStreamConsumer(flushQueueToDestination):181 - Flushing buffer: 26208887 bytes2022-03-24 11:54:43 destination > 2022-03-24 11:54:43 INFO i.a.i.d.b.BufferedStreamConsumer(lambda$flushQueueToDestination$1):185 - Flushing zendesk_tickets: 1583 records2022-03-24 11:54:43 destination > 2022-03-24 11:54:43 INFO i.a.i.d.r.RedshiftSqlOperations(insertRecordsInternal):43 - actual size of batch: 15832022-03-24 11:54:43 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):301 - Records read: 53000 (204 MB)2022-03-24 11:55:02 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):301 - Records read: 54000 (208 MB)2022-03-24 11:55:15 destination > 2022-03-24 11:55:15 INFO i.a.i.d.b.BufferedStreamConsumer(flushQueueToDestination):181 - Flushing buffer: 26212538 bytes2022-03-24 11:55:15 destination > 2022-03-24 11:55:15 INFO i.a.i.d.b.BufferedStreamConsumer(lambda$flushQueueToDestination$1):185 - Flushing zendesk_tickets: 1713 records2022-03-24 11:55:15 destination > 2022-03-24 11:55:15 INFO i.a.i.d.r.RedshiftSqlOperations(insertRecordsInternal):43 - actual size of batch: 1713Jean Sahlberg
03/24/2022, 12:54 PMthe log just stops there
a
Augustin Lafanechere (Airbyte)
03/24/2022, 4:43 PMHi @Jean Sahlberg do you know if the source contains more than 54000 records? I'd like to check if the problem comes from a partial read or a partial write 😄
j
Jean Sahlberg
03/24/2022, 4:51 PMHi @Augustin Lafanechere (Airbyte), thanks for your reply. Yes, it contains more than 54000 records, it have over 300k records.
a
Augustin Lafanechere (Airbyte)
03/24/2022, 4:53 PMCould you check if the container's memory is not full while running the sync (source + worker + destination)?
j
Jean Sahlberg
03/24/2022, 5:02 PMsure, I will do that and reply back!
Jean Sahlberg
03/24/2022, 6:07 PMno, the container have plenty memory and the sync just halts… now I’m running it on a EC2 machine to make sure is not MacOS the problem…
j
Jean Sahlberg
03/24/2022, 7:13 PMok. thanks
4 Views