Hi :slightly_smiling_face:, I have a question conc...
# replication-troubleshooting
a
Hi 🙂, I have a question concerning the MongoDB source connector. We are only syncing one stream out of many collections but we are experiencing latencies because of many queries done on all the collections. From what I’ve found, it seems to be because of the discovery phase which requests all collections to fetch their schema. Are my suspicions accurate ? If so, is it possible to only discover the streams which are only checked for syncs ? We are using Airbyte version 0.40.29 with Mongo source connector 0.1.19, and we schedule the sync with Dagster every 15 minutes.
m
You’re correct. Today there are some issues https://github.com/airbytehq/airbyte/issues/14246 reporting the provlem with large collections/datasets with the Source Mongo connector. There isn’t a ETA to fix the issue 😞
a
Alright, thank you for the answer !
Something I don’t understand tho is why it needs to fetch the schema of all the collections at sync time. Should not it be better to only fetch the collection that are being synced ?
u
Probably it is because Mongo today is alpha and there are a lot of improvements to made in the connector!