Hi Team, I have an interesting scenario. New airby...
# replication-troubleshootinge
Eric Hartono
11/22/2022, 9:59 PMHi Team, I have an interesting scenario. New airbyte instance running in EC2 on t3.large instance.
Here’re my notes.
• Github to Redshift connection
• Airbyte v0.40.18, connectors use latest (updated yesterday)
• Replication config is the default recommended config
• Full Sync took 1hr 20m
• Next sync took 1hr 19m
I expect next sync (there are few tables with incremental dedupe sync) to be much faster, but it’s not the case. Is there any reason why this is not faster?
Thanks.
m
Marcos Marx (Airbyte)
12/02/2022, 5:30 PMCan you check how much time normalization is taking? Maybe look the sync summary and identify the stream took more time to finish
e
Eric Hartono
12/04/2022, 10:40 PM@Marcos Marx (Airbyte) It’s around 38 mins for sync on screenshot above.
Eric Hartono
12/04/2022, 10:40 PMRecent sync from overnight, it’s around 32 mins.
Is that expected taking that long? Thanks @Marcos Marx (Airbyte)
m
Marcos Marx (Airbyte)
12/05/2022, 2:19 PMIf you have full refresh streams it will always recreate the tables. You can check from 176 tables created which one is consuming more time
👀 1
2 Views