https://linen.dev logo
Join Slack
Powered by
I see Intercom to Redshift sync is doing a ton of ...
# ask-community-for-troubleshooting
t
I see Intercom to Redshift sync is doing a ton of stuff, saying it’s already read 791k records but I only see one file in S3 for our companies, and when I open the file it has 295k companies which matches what we have on Intercom. What exactly is the sync doing? There are 296k log messages so I’ll just post the latest.
👀 1
Copy code
2022-01-19 15:26:48 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:48 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 48 containing 10.01 MB]
2022-01-19 15:26:49 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:49 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 49 containing 10.01 MB]
2022-01-19 15:26:49 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:49 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 50 containing 10.01 MB]
2022-01-19 15:26:49 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:49 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 51 containing 10.01 MB]
2022-01-19 15:26:49 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:49 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 52 containing 10.01 MB]
2022-01-19 15:26:49 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:49 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 53 containing 10.01 MB]
2022-01-19 15:26:50 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-01-19 15:26:50 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - {} - [Manager uploading to 083950260099-ck-luigi/airbyte/airbyte/intercom_contacts/2022_01_19_1642555399775_adbbc01f-5f03-45fd-ac33-956503546d49.csv with id 47_ZuFMX4...RPTUqXVMJ]: Finished uploading [Part number 54 containing 10.01 MB]
2022-01-19 15:27:33 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 785000
2022-01-19 15:28:24 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 786000
2022-01-19 15:29:14 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 787000
2022-01-19 15:30:01 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 788000
2022-01-19 15:30:54 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 789000
2022-01-19 15:31:44 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 790000
2022-01-19 15:32:38 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 791000
2022-01-19 15:33:30 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 792000
2022-01-19 15:34:24 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 793000
2022-01-19 15:35:15 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 794000
2022-01-19 15:36:06 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$2):216 - Records read: 795000
I don’t even see an intercom_contacts directory despite it saying that it’s finished uploading
I only see the intercom_companies directory
m
@Tyler Buth the sync is running and some data is stored in temporally dirs; did the sync finish now?
t
It looks like the syncs are finishing every time, records are being stored, the only consistent error is with page 667 of companies, other errors appear to be only sometimes.
m
but the number of records in destination are correct now?
6 Views
Open in Slack
PreviousNext
Powered by