Hello fellow Airbyters! This is a Facebook question / request for information I have a dev app that connects to business app on Facebook. This dev up was built specifically for Airbyte and nothing else is using it. I am pulling data only from ads_insights. I already asked and was approved by Facebook for Advanced Access. The app review was approved. Business was verified. I am nowhere near the application-level rate limit, here in FB dashboard I am almost always at 100% remaining(even when I am throttled). I am running updated Airbyte version 0.40.18 data is going into GCP - BQ every 2 hours. With Loading method as GCS Staging. Source Settings: Start date: 2022-01-28T000000Z End date: blank Include Deleted: off Fetch Thumbnail Images: off Page Size of requests: 10 Insights Lookback Window: 7 Maximum size of Batched Requests: 50 Airbyte is located on Compute Engine with 2 vCPUs and 8 Gigs of ram and 120 Gigs of disk space. Why am I still throttled? It takes over 1 hour and 20 minutes to transfer 80mb of data where my Postgress instance does 200 mb in less than 4 mins. Any help, information, or feedback is very much appreciated.

Hi, Oleg! Have you seen these docs on tweaking the Airbyte configuration? https://docs.airbyte.com/operator-guides/configuring-airbyte/#jobs https://airbytehq.github.io/operator-guides/scaling-airbyte/ I'll look into this more though, so hang tight!

Hey, Nataly! Thank you for looking into this🙏. Do let me know if there is something else I can do here. My most recent sync just run for over 2 hours 😑

I see! Could you please give me the logs that you are getting?

Hi Nataly! Here are the logs for that 2 hour sync session. I hope you can point to the right direction here. Thank you, have a great weekend.

hey @Nataly Merezhuk (Airbyte), any update regarding this?

Hi Oleg, thanks for your patience - our small team is still catching up after Hacktoberfest! So if you look in the logs you'll see things like

Utilization is too high (90)%, pausing for 1 minute

I found this issue that's quite old but related: https://github.com/airbytehq/airbyte/issues/8773 One of the comments is really insightful: "the rate limit is dynamically computed according to the number of ads on the user account. What they are calling the Business Use Case Rate Limits which is documented here." I think the current workaround would be to split this into multiple connectors. This is in the backlog to fix, but I don't have an ETA to give you yet.

Thank you Nataly!

No problem!