I had a quick question over replication - Does Air...
# advice-data-ingestionu
Umang Sheth
08/23/2022, 9:36 PMI had a quick question over replication - Does Airbyte support parallel replication, when we choose multiple streams for a single source? MySql -> BigQuery
m
Michel (Airbyte)
08/23/2022, 11:13 PMHey @Umang Sheth we don’t today. we needed https://github.com/airbytehq/airbyte/issues/14852 that we just released. This change will allow us to support what you’re asking for (although we still need to implement the feature). In the meantime, what we’ve seen other community members do is to create multiple connectors with subsets of streams
z
Zaza Javakhishvili
08/24/2022, 5:38 AM@Michel (Airbyte)
I have about 700 connections because of this.
One source connector is used only for 17 connections, but anyway looks like they are not running in parallel.
It is running (showing) in parallel but it is very slow.
I am monitoring the server and it has enough resources...
What can be the reason?
Zaza Javakhishvili
08/24/2022, 3:35 PM@Michel (Airbyte)
Here you can compare execution time.
When it runs with other connections (same time) and when I run a single one.