Hey guys, been converting a current monolithic app into a Serverless one using SST. The database is mongo and we want to leave it as is for first phase. What is the best practice for connecting to an atlast hosted mongo cluster? Is the index method inside the stacks folder run inside a docker container? Can I cache the DB connection in there or is there a better way?

Hey @Matthew Smithburger, I haven’t used Mongo extensively, but can you check if this example helps https://serverless-stack.com/examples/how-to-use-mongodb-in-your-serverless-app.html

This is how I was using mongodb via mongoose. Here’s the mongo-client.ts :-

import mongoose from 'mongoose';

const uri = process.env.MONGODB_URI;

if (!uri) {
  throw new Error('Error connecting to database');
}

let cachedPromise: Promise<mongoose.Connection> | null = null;

export const connectToDb = async (): Promise<mongoose.Connection> => {
  if (!cachedPromise) {
    cachedPromise = mongoose.createConnection(uri, {
      useNewUrlParser: true,
      useCreateIndex: true,
      useUnifiedTopology: true,
      useFindAndModify: false,
      connectTimeoutMS: 5000,
      bufferCommands: false,
      bufferMaxEntries: 0,
      poolSize: 20,
      autoIndex: false,
    });
  }
  const connection = await cachedPromise;
  return connection;
};

And here’s how the lambda function would use it :-

import type { APIGatewayProxyEvent, Context, APIGatewayProxyResult } from 'aws-lambda';
import middy from '@middy/core';
import httpHeaderNormalizer from '@middy/http-header-normalizer';
import httpErrorHandler from '@middy/http-error-handler';
import { Static } from 'runtypes';

import { IGenre } from '../../types/Genre';
import GenreSchema from '../../schema/Genre';
import { connectToDb } from '../../utils/mongo-client';

type Genre = Static<typeof IGenre>;

const baseHandler = async (event: APIGatewayProxyEvent, context: Context): Promise<APIGatewayProxyResult> => {
  context.callbackWaitsForEmptyEventLoop = false;
  const query = { isActive: true, forLoLoMo: true };
  const connection = await connectToDb();
  const Genre = connection.model<Genre>('Genre', GenreSchema);
  const genres = await Genre.find(query, [], { sort: { sortOrder: 1 } });
  return { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(genres) };
};

export const handler = middy(baseHandler);

handler.use(httpHeaderNormalizer()).use(httpErrorHandler());

Hope this helps. This used to give me around 350 - 800 ms response times on cached connections. Around 1s to 1.2s on new connections and 1.5 to 1.8s on cold starts. You could try atlas vpc peering to reduce the connection time or you could use mongodb serverless which is in beta. You can’t use mongoose with mongodb serverless though.

@Ashishkumar Pandey very helpful, will try this, thanks you!

@Ashishkumar Pandey are you not using a VPC peering connection? Curious how you're setting up your networking for connection within your VPC? @bike-bill

@William Hatch This particular API is in a public VPC and so no peering necessary.

@Ashishkumar Pandey ah, I see. Do you know if there's a different connection string to use when you're peered? Atlas provides the url, which would seem as if anything trying to connect to it from inside the vpc would have to make a request through the vpc nat gateway, but, in the function logs, we're seeing the local private IP addresses that coincide with the atlas cidr range. I'm assuming AWS must be doing some translation, or something. I'm by no means a network specialist, but given the issues we're having, looking at anything that could help explain them. Thanks for the reply.

@William Hatch check this out. This was the guide that I had used to setup peering a few years ago when I had to use a private VPC. The way peering works is that both networks need to be able to discover or know about each other. So, here you inform Atlas about your AWS VPC by providing all info about your AWS VPC that the peering connection wizard asks for and then you inform AWS VPC about Atlas network by adding Atlas network info into the AWS VPC routing table. Since, the AWS and Atlas networks now know about each other, they’ll create a tunnel which basically will have local address range that’ll forward traffic to each other. It has been some time since I’ve visited this so some things could’ve changed but the fundamentals would remain the same. If you could go in detail about the issues that you’re facing I might be able to narrow down some solutions.

Thanks, @Ashishkumar Pandey. We validated the peering with Atlas TS yesterday, but, I'd just thought that maybe there's a different connection string when using one, and wasn't sure. This link https://docs.atlas.mongodb.com/reference/faq/connection-changes/#std-label-atlas-faq-custom-dns answers all that. Our current issue is not being able to reliably get connections; will work fine for a while, then connection timeout failures across all functions that connect to Atlas. I just posted more details in the help channel https://serverless-stack.slack.com/archives/C01JG3B20RY/p1643915030447699 Thanks again.