I am doing Mixpanel to Redshift migration, based on the logs, I see every 1000 records are being moved as a batch (which is slow). How to increase this number to make data load faster?
Sharath Chandra
02/03/2023, 7:06 AM
@Marcos Marx (Airbyte) Could you please help? Also I need to increase the API rate limit time. Any idea?
u
user
02/03/2023, 9:37 PM
You can try to use the staging method which should be faster to load the data to Redshift. Today Airbyte hardcode the batchsize to 1000 records