Hi I’ve started to test something locally. I trie...
# ask-community-for-troubleshooting
a
Hi I’ve started to test something locally. I tried adding a source which is a MongoDB Atlas cluster in our dev environment. I’m able to connect to it from my local programmatically. I have tried different things. Including specifying the mongo replica set. additional details: we’re behind a cloud VPN. Might it affect things as airbyte is containerized? log text:
Copy code
DefaultAirbyteStreamFactory(lambda$create$0):73 - W, [2021-05-14T16:07:49.870271 #8] WARN -- : MONGODB | Error running ismaster on <redacted server name>.<http://mongodb.net:27017|mongodb.net:27017>: SocketError: getaddrinfo: Name does not resolve
2021-05-14 16:08:50 INFO (/tmp/workspace/c0bdcfee-0a4d-48c0-8de1-560e9908d59b/0) DefaultAirbyteStreamFactory(internalLog):110 - [2021-05-14 16:07:50 +0000] WARN : | "MONGODB | Error running ismaster on <redacted server name>.<http://mongodb.net:27017|mongodb.net:27017>: SocketError: getaddrinfo: Name does not resolve"
2021-05-14 16:08:50 INFO (/tmp/workspace/c0bdcfee-0a4d-48c0-8de1-560e9908d59b/0) TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
Help is greatly appreciated 🙂
👀 1
u
hi @Adhithya Ravichandran which connector version are you using? we just update mongo to support replica set
a
I have the version which supports replica set 🙂
s
Hey adhithya, this sounds like potentially a networking Issue. Where is Airbyte running and where is Mongo running? Is there a need to whitelist any ips or something similar?
a
nope it’s running on my local and i’ve been able to connect to the same server through Mongo Compass as well as javascript and python connectors
through VPN
sorry I didn’t update earlier. @[DEPRECATED] Marcos Marx helped me out and I was able to make local changes and get a dev build of the connector to work. The issue was around how the connection string was being composed together
🎉 1
I’ll try and write a PR but I think my fix is a bad hack
s
thanks @Adhithya Ravichandran, would love to understand the issue and maybe find a non hacky fix 😛
u
@Adhithya Ravichandran i dont think so! as I said to you, if this solved your problem the solution can be very useful to us understand mongo or how to build the connector in a better way =D
a
happy to help with that. how should we coordinate? the issue was this: https://github.com/airbytehq/airbyte/blob/ada39b74064b48e5505d8b42c3b509a7b9bd9a2f[…]te-integrations/connectors/source-mongodb/lib/mongodb_source.rb I changed it locally to
Copy code
uri = "mongodb+srv://#{@config['user']}:#{@config['password']}@#{@config['host']}/#{@config['database']}?authSource=#{@config['auth_source']}"
                  if !@config.fetch(:"replica_set", "").strip.empty?
                    uri += "&replicaSet=#{@config['replica_set']}&ssl=true"
                  elsif ["true", true].include?(@config.fetch("ssl", false))
                    uri += "&ssl=true"
                  end
the change was the addition of
+srv
and removing the port number. in our config of mongo atlas, host and port number from any incoming connections is automatically resolved. However the connection string should follow this format
OK @[DEPRECATED] Marcos Marx just getting off work, I’ll open a PR tomorrow morning 😅
airbyte heart 1
oh I also didn’t toggle the TLS switch when connecting, I think TLS is used by default in our config. I think the most elegant solution would be to ask the user if they wanted to provide a connection string and forgo all the hassle of inputting user, password etc if they say yes… this is how Mongo’s own Compass desktop app does it
done. let the scrutiny begin 😅