Good Evening everyone. I am having problems connecting my MongoDB Atlas database as a source at airbyte. Currently running opensource @docker locally at my machine (tried many times and always have the same error Failed to fetch mongodb schema). I am able to connect properly, but receive this error when building the pipeline. The whole database is big (around 60GB). Any ideas of what can be done?

@[DEPRECATED] Marcos Marx turned this message into Zendesk ticket 2686 to ensure timely resolution!

Hi! Thank you for your patience, I was out of the office yesterday. I need more details to help you figure this out, so here are a few questions: 1. Could you please double check that you've followed all the steps to configure your database for a connection with Airbyte? Sometimes these kinds of issues can arise due to incorrect configuration or authentication issues. Docs: https://docs.airbyte.com/integrations/sources/mongodb-v2/#getting-started 2. Could you provide any possible error logs that you're getting? Ones from Airbyte but also any logs you're getting from MongoDB? This could really help figure this out.

Hi! Thank you for your patience, I was out of the office yesterday. I need more details to help you figure this out, so here are a few questions: 1. Could you please double check that you've followed all the steps to configure your database for a connection with Airbyte? Sometimes these kinds of issues can arise due to incorrect configuration or authentication issues. Docs: https://docs.airbyte.com/integrations/sources/mongodb-v2/#getting-started 2. Could you provide any possible error logs that you're getting? Ones from Airbyte but also any logs you're getting from MongoDB? This could really help figure this out.

2022-10-09 160008 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.host: is missing but it is required, $.port: is missing but it is required, $.instance: does not have a value in the enumeration [standalone] 2022-10-09 160008 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.instance: does not have a value in the enumeration [replica], $.server_addresses: is missing but it is required

Could you please provide the full logs you are getting? I can't help you debug from the partial ones you've provided above. Thanks!

server-logs (2).txt,server-logs (1).txt,server-logs.txt

Could you please double check your MongoDB configuration - Is the IP address of your Airbyte instance set in the IP access list?

Could you please double check your MongoDB configuration - Is the IP address of your Airbyte instance set in the IP access list?

yes! I have removed the restrictions

Could you try

0.0.0.0,::

? That will help if you're using an IPv6 address!

@Nataly Merezhuk (Airbyte) tried inputting the above mentioned ip and got the following error

Could you please try connecting to MongoDB from another source - maybe the terminal? Just to narrow down where the connection issue is stemming from.

I am connected to mongo using the same credentials on many other apps with no problems. I have read on another article that someone faced the same problem because of special characters in the data- can this be the case? We use parse server as a middle layer that connects to mongodb and generate all the relations between tables etc and theses relations are represented by the objectID$refTABLE

That is an excellent observation. Could you please link the article that you were reading?

Thanks for linking all the resources you've found! I am assuming increasing Docker resources isn't fixing the issue for you?

correct, didnt solve my issue

Hey Sergio, thanks for your patience! This is an intricate issue to track down. I've found an open issue on GitHub: https://github.com/airbytehq/airbyte/issues/14987 I also found this MongoDB doc that may provide a workaround for the special character: https://www.mongodb.com/docs/manual/reference/operator/aggregation/literal/ As it stands, I believe that issue needs to be resolved before there is an official fix for this, unfortunately!

@Nataly Merezhuk (Airbyte) how are you? Just as a follow up to this thread, I have just started a free trial @ airbyte cloud and was able to connect my mongodb instance and transfer data to big query without any problems

Good to know, thanks for the update!

@Sergio Spieler Hey I’m running into the same issue. I just upgraded from t2.medium to t2.Large (8GB Ram, 4vCPU). Did the upgrade work for you?