Is there a way to limit the traffic during sync so...
# ask-community-for-troubleshooting
j
Is there a way to limit the traffic during sync so that Airbyte causes minimum impact to the source DB?
1
d
What do you mean by limiting traffic? Limiting the number of connections?
j
I mean some sort of rate limiting to control the throughput. If I have a connection containing 100 tables, would Airbyte spin up multiple workers to sync them as fast as possible? If so, it may cause performance impact to the source DB. I didn’t find any documentation about whether a connection would sync multiple tables in parallel or not, except this article https://docs.airbyte.io/operator-guides/scaling-airbyte mentions:
Copy code
Jobs-wise, we are mainly concerned with Sync jobs when thinking about scale. Sync jobs sync data from sources to destinations and are the majority of jobs run. Sync jobs use two workers. One worker reads from the source; the other worker writes to the destination.
It sounds like for a sync job (in a connection), only 1 worker reads from the source db (so 1 table a time?) and as a result I don’t need to worry about the performance impact to the source. Can you confirm my understanding is correct?
d
Yes. This worker is single threaded, and should only use 1 connection. However, if there are multiple connections to the same database with jobs running at the same time, too many connections can be used
j
make sense, thank you🙏