is there a recommendation on how many tables per c...
# ask-community-for-troubleshootingg
gunu
08/18/2021, 1:30 PMis there a recommendation on how many tables per connector? i.e. if i have 20 tables of varying sizes from one source. should i create one connector with all tables or break it into seperate connectors from the one source (and run them at different times).
additiionally, there would be a few sources with this situation.
so many connectors with few tables? few connectors with all the tables from each source?
✅ 1
o
Opeyemi Fabiyi
08/18/2021, 1:50 PM@gunu
This was what I did
Opeyemi Fabiyi
08/18/2021, 1:51 PMI had over 150 tables to migrate so I broke it into 10 different connections because I observed when I use 1 connection it kept breaking
g
gunu
08/18/2021, 2:01 PMI’d be curious as to why it would break if it’s chunking records into batches. it would take longer for one connector but more likely to break if multiple connectors running and the same time no?
u
[DEPRECATED] Marcos Marx
08/18/2021, 6:18 PM
I saw some users splitting one connector for small tables (they will sync fast and if something failed is easy to catch up) and individual connector for large tables. Because if you have a table that took 6h is best try to parallelize the proccess.
👍 2
j
Jeff Crooks
08/20/2021, 11:28 AMsomething similar on my side, I split full/incremental for some sources in different conections
2 Views