https://serverless-stack.com/ logo
Join Slack
Powered by
Hey guys, anybody using local development with RDS...
# sst
a
Hey guys, anybody using local development with RDS connected with SSH? I have to use a jump-box to connect to my RDS database, and want to setup this with SST for local development.
t
Are you using serverless RDS?
a
I believe so?
t
What library are you using to connect to rds from node
a
I’m using TypeORM.
Maybe I can modify the connection string to have SSH? 🤔
Well we use public key for auth, not sure.
t
My suggestion would be to use ssh port forwarding
a
Ok.
Any tutorial you have in mind?
t
you can forward your localhost:5432 to point to the remote RDS, then your library can just connect to localhost:5432 like nothing is different
You can also checkout this: https://github.com/sshuttle/sshuttle
a
Ok checking.
But this requires me to modify the jump box? or it’s just client side?
t
all you need is ssh access I believe
a
Ok yeah, makes sense, going to check it.
t
I'm putting together recommendations for RDS + Serverless and in general I'd recommend using the Data Api so that you don't have to worry about vpns
Can just access it using your normal aws auth
a
Yes but data API works with ORM?
With TypeORM like.
t
There actually is a data-api driver for typeorm: https://www.npmjs.com/package/typeorm-aurora-data-api-driver We just built a data api integration for this library as well: https://github.com/koskimas/kysely https://github.com/serverless-stack/kysely-data-api
a
Interesting, going to check it.
Data API is only for Aurora Serverless?
I don’t understand the difference yet.
This is what I use.
I don’t have much experience with RDS/Aurora/etc… somebody setup this in my organization.
t
Yeah so RDS has a bunch of different flavors
a
Still trying to wrap my head around it.
t
Data Api only works with Aurora Serverless (the rest of the options are a traditional spin up a node, specify ram + cpu)
a
Ok.
t
Serverless will scale up and down transparently (tbh not that well) and has an HTTP based api so functions don't have to deal with connection pooling
a
How I can create aurora serverless?
Yeah nice.
t
You can do it through CDK, I don't have the commands off the top of my head but I'm going to be doing some benchmarking this week so I'll be spinning this stuff up myself
a
Ok cool.
Yeah looking into the Console now.
To see if I can create one for testing.
Looks like this is the key.
In the setup for a new DB.
t
yep looks right
a
What should I use for version?
t
🤷🏽 up to you haha
Latest and greatest!
a
Just wondering if data api was available on both.
Based on some other comments.
s
I'm in a similar boat as @Adrián Mouly re: RDS. I'm working with a legacy RDS DB (non-serverless flavor) and I'm in the process of migrating the service from Serverless Framework to SST. I'm still wrapping my head around best practices related to RDS, specifically spinning up an ephemeral RDS DB for development. I'm used to working with DynamoDB where creation/deletion in an SST dev environment is super easy. RDS requires me to set up an EC2 instance, VPC access, security groups, subnets, blah blah blah. 🤯 Looking forward to reading about your recommendations re: RDS @thdxr, because this is painful 🙂
t
👍🏽 yeah thinking through some of that now as well. Hopefully can find a happy path for all of this
a
@Seth Geoghegan I don’t think I needed to setup EC2 in my case.
s
my happy path is convincing folks to use DynamoDB, but I'd settle for a less painful way to integrate with RDS in the meantime
@Adrián Mouly it looks like you were using Aurora, which I believe is the Serverless version?
a
Maybe…
I don’t know how to know it, hahaha.
s
Yeah, me either 🤷
a
I don’t see in the Console a way to know if my instances are serverless or not.
Due this was setup before me, hard to know.
s
Yeah, in my case, the Serverless Framework app I hope to replace just imports an existing RDS that was provisioned....somehow?
so I don't exactly have an example of how it was all setup, so I go console surfing to understand it all
a
Yeah me too.
My case is similar, the RDS it’s also provisioned from SLS.
t
It probably isn't a serverless instance
a
This is kinda hidden, haha.
t
Yeah had trouble finding that myself
a
@thdxr with normal RDS you can have Write/Read endpoints, with Serverless you also can?
@Seth Geoghegan look, the one on the top is Aurora Serverless.
Only way to know the difference is the Size.
Aurora Serverless has 8 capacity units (left default).
Also in details it says.. Role: Serverless.
s
So, this may be dumb and non-sensical, but... would it be possible to run Aurora Serverless in development with SST while deploying to the non-serverless RDS Postgres instance for staging/prod?
for no other reason than setting up an RDS database is a huge PITA and I just want something I can spin up/throw away as needed.
a
@Seth Geoghegan yes, planning to do that.
I want to do this: dev/stage -> use Aurora serverless prod -> use RDS Aurora (normal)
Is that what you mean?
s
Well, I'm not using Aurora in prod since I've inherited a legacy setup
a
Ah ok.
s
I just want a serverless development experience
a
I’ve inherited Aurora RDS in prod 😂 . But not serverless.
Yes, you can setup Aurora Serverless with CDK, and use that ONLY for dev.
s
haha, enough paid to go around!
a
It’s just an IF in your stack definition.
And then in your code, you might use different driver.
That’s my goal at the moment… to use Aurora Serverless in DEV, and the Lambda should be agnostic, maybe just switch the driver to use Data API.
Not sure if there is any concern on doing that.
s
yeah, I'm gonna think on this. I really don't like the experience of spinning up an RDS Postgres DB in development. I'm spoiled on DynamoDB
a
Haha.
I have like 50 sql tables.
Too late for me.
Hhahaha.
Also have some Dynamo too.
s
💀🪦💐
welp, I guess one of the benefits of live local lambda development is that I can connect to Postgres running locally in a docker container from my lambda. That's a much better experience than waiting for an RDS db to be provisioned in AWS.
a
😮
You just put the local connection string right?
I might do that too.
s
Yup, it just dawned on me that it may be an option
tried it out, and it worked
a
Yeah sounds reasonable.
s
which is a huge relief
a
My problem is trying to use RDS cloud instance from local, and is not possible due VPC.
s
yeah, it's painful
a
I have a huge DB in dev (an snapshot of prod).
t
Yeah I've been debating local docker vs personal rds
Some stuff is annoying with personal rds
s
Setting up RDS using CDK was way too much friction for a local dev experience. I'd be interested to see if snapshots are any faster, but it took minutes to spin up the infrastructure required to use an RDS db.
t
It's only a one time cost though right?
I also was thinking to run the personal one outside a vpc
s
I suppose it would be a one time cost, but I found the entire setup to be laborious. My preference for serverless is so strong, I can't even with this RDS shenanigans 😆
k
FYI, the RDS serverless CDK snippet I have so far..
Copy code
// Create the VPC needed for the Aurora Serverless DB cluster
    const vpc = new ec2.Vpc(this, "MainVPC");

    // Create the Serverless Aurora DB cluster
    const cluster = new rds.ServerlessCluster(this, "ResourceServiceCluster", {
      vpc,
      defaultDatabaseName,
      // Set the engine to MySQL
      engine: rds.DatabaseClusterEngine.AURORA_MYSQL,
      parameterGroup: rds.ParameterGroup.fromParameterGroupName(
        this,
        "ParameterGroup",
        "default.aurora-mysql5.7"
      ),
      // Optional, disable the instance from pausing after 5 minutes
      scaling: { autoPause: cdk.Duration.seconds(0) },
    });
Open in Slack
PreviousNext
Powered by