https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
g

gunu

08/18/2021, 1:30 PM
is there a recommendation on how many tables per connector? i.e. if i have 20 tables of varying sizes from one source. should i create one connector with all tables or break it into seperate connectors from the one source (and run them at different times). additiionally, there would be a few sources with this situation. so many connectors with few tables? few connectors with all the tables from each source?
1
o

Opeyemi Fabiyi

08/18/2021, 1:50 PM
@gunu This was what I did
I had over 150 tables to migrate so I broke it into 10 different connections because I observed when I use 1 connection it kept breaking
g

gunu

08/18/2021, 2:01 PM
I’d be curious as to why it would break if it’s chunking records into batches. it would take longer for one connector but more likely to break if multiple connectors running and the same time no?
u

[DEPRECATED] Marcos Marx

08/18/2021, 6:18 PM
I saw some users splitting one connector for small tables (they will sync fast and if something failed is easy to catch up) and individual connector for large tables. Because if you have a table that took 6h is best try to parallelize the proccess.
👍 2
j

Jeff Crooks

08/20/2021, 11:28 AM
something similar on my side, I split full/incremental for some sources in different conections
2 Views