https://serverless-stack.com/ logo
#guide
Title
# guide
p

Patrick Gold

10/24/2021, 11:47 PM
@thdxr - would love to pick your brain about Prisma + Aurora + SST. I have a few questions if you get a minute to reply. Thank you! 1. Are you configuring SST’s APIs to hit a local database when running
sst start
? If so, how is that configured? 2. How are you handling Prisma client generation and migrations inside of a CI/CD pipeline? (I’m using Github Actions) 3. I haven’t gotten to the Prisma layer deployment yet, but anything tricky to note there?
t

thdxr

10/24/2021, 11:52 PM
I don't use Prisma anymore but back when I was using it I did use a local postgres. I basically passed a conditional parameter into the lambda function like this:
Copy code
DATABASE_MODE: app.local ? "local" : "remote"
And then it would only connect locally during
sst start
2. Generally I like gitignoring anything generated with codegen and do everything in CI. Prisma by default writes into
node_modules
so it's likely ignored. I just ran
yarn prisma generate
as a step before building
3. Shouldn't be anything tricky with the Prisma layer. It is big which will slow down your cold starts
I recommend looking into Prisma's data proxy. It makes it so you don't ship these heavy binaries inside your lambda but it's another thing to configure
p

Patrick Gold

10/24/2021, 11:55 PM
Curious what you’re using in favor of Prisma now?
t

thdxr

10/24/2021, 11:56 PM
I've been doing a bunch of prototyping and benchmarking with this: https://github.com/koskimas/kysely My current recommendation is to use that library with Planetscale if you want a relational db. If you want to stay in AWS recommend using RDS + Data API. Wrote a bit about the benchmarking here: https://thdxr.com/post/serverless-relational-showdown
p

Patrick Gold

10/25/2021, 12:02 AM
I have to stick with AWS, but Planetscale looks cool. My biggest issue with Data API is there’s not a lot of tooling ecosystem support for it.
I see you’re building something though 😄 https://github.com/serverless-stack/kysely-data-api
t

thdxr

10/25/2021, 12:05 AM
Haha yeah integration with kysely and data api. It's actually complete, just need to write up some readme and add some more tests
That's why I built it - data api is inaccessible right now but it is the least annoying way to use RDS with serverless
p

Patrick Gold

10/25/2021, 12:06 AM
Curious how you’d handle migrations via the data api without this adapter for kysely.
t

thdxr

10/25/2021, 12:18 AM
kysely supports migrations and the adapter I have works with it
But otherwise I maybe would do migrations using a tool going over the normal connection. I really avoid VPC though so I'd probably just use data api
d

Damjan

10/27/2021, 2:28 PM
Planetscale doesn’t have read replica binlog support so it’s not easy to get data out for analytics
p

Patrick Gold

10/27/2021, 4:22 PM
@thdxr I’m back at data-api and wondering how you’ve done local testing in the past. I found this package: https://github.com/koxudaxi/local-data-api - Curious if you have any other solutions to using data-api with a local database.
t

thdxr

10/27/2021, 4:23 PM
I've been abstracting database access through
kysely
so for local mode I just use the normal postgres adapter. No part of my application is aware if it's talking to data api or not
p

Patrick Gold

10/27/2021, 4:23 PM
oooh that makes sense
t

thdxr

10/27/2021, 4:24 PM
Otherwise I'd recommend always using a remote RDS instance even for "local" dev
p

Patrick Gold

10/27/2021, 4:25 PM
Even for tests?
t

thdxr

10/27/2021, 4:26 PM
Yeah, it's weird but serverless is a big shift so a lot needs to be rethought. For dynamo I use a remote instance for everything
I'm even thinking about running my tests remotely 😱
p

Patrick Gold

10/27/2021, 4:26 PM
How confident are you with your data-api adapter for kysely? I’d love to use it but yanno, kinda fresh 😄
data-api-client leaves much to be desired
Is SST officially supporting that adapter? I noticed it’s under the org
t

thdxr

10/27/2021, 4:28 PM
Yeah we're writing a new getting started guide and for the RDS path we're going to be using kysely + our adapter. That said you can look to see how little the adapter is actually doing, just forwarding the built sql to the data api and processing responses
So really comes down to how good kysely itself is
And kysely is the best typescript codebase I've seen in my life so I have a lot of faith in the author. It is relatively new and not as well known though
p

Patrick Gold

10/27/2021, 4:29 PM
yeah that’s my biggest concern. It does look great though
I feel better knowing that SST will be adopting it though
Giving Kysley a go, but encountering an error when transpiling to lambda code:
Copy code
> node_modules/pg/lib/native/client.js:4:21: error: Could not resolve "pg-native" (mark it as external to exclude it from the bundle, or surround it with try/catch to handle the failure at run-time)
    4 │ var Native = require('pg-native')
You know how to get around that?
t

thdxr

10/27/2021, 5:12 PM
yeah need since pg has native deps it can't be bundled
in your function
Copy code
bundle: {
        nodeModules: ["kysely", "pg"],
      },
p

Patrick Gold

10/27/2021, 5:13 PM
ah, yes. will give that a try ty
Hmm still complaining about pg-native. Do I need to explicitly put pg-native in there?
I guess I didn’t have pg-native installed
only pg
t

thdxr

10/27/2021, 5:26 PM
I didn't think I needed pg-native
I guess try it and see
p

Patrick Gold

10/27/2021, 5:26 PM
yeah it’s working now
Got it working with your dialect!
Sweet
Soooo much better than data-api-client
t

thdxr

10/27/2021, 5:33 PM
nice 🙌🏽
p

Patrick Gold

10/27/2021, 6:13 PM
Hmmm... In my CI runner it fails to bundle and it's def in my package.json. Building Lambda function packages/backend/src/services/default.handler Error: Cannot extract version for module 'kysely'. Check that it's referenced in your package.json or installed. at extractDependencies
never seen that error before with any other modules. something special about the bundle key? Need to read the docs I guess. 😅
t

thdxr

10/27/2021, 6:41 PM
Are you using a mono-repo setup?
I think with our currently implementation you have to add kysely to your package.json at root.
bundle.nodeModules
is basically a list of packages that are pulled from npm and added into your function's bundle. So it looks for at package.json to find the version. We should probably just copy it from
node_modules
cc @Frank does that work or is there a reason we pull from npm?
p

Patrick Gold

10/27/2021, 6:46 PM
Yes, using yarn workspaces
t

thdxr

10/27/2021, 6:46 PM
Yeah if you add kysely + other nodeModules to the root package.json it should work for now. We should make this a bit smarter so it finds the nested package.json
p

Patrick Gold

10/27/2021, 6:48 PM
This is only for packages in bundle?
t

thdxr

10/27/2021, 6:48 PM
yeah
right now it looks for the package version in
package.json
next to
sst.json
- which is why it's complaining it can't find the version
p

Patrick Gold

10/27/2021, 6:49 PM
Ah, makes sense
t

thdxr

10/27/2021, 6:49 PM
Really it should be looking in yarn.lock if anything but we should probably not care about version and just copy it out of your node_modules
p

Patrick Gold

10/27/2021, 6:49 PM
right, right
f

Frank

11/02/2021, 2:17 AM
yeah if u specify
bundle.nodeModules
, SST creates a temp folder, write a
package.json
with the modules specified in
bundle.nodeModules
, and then does a npm/yarn install. This to ensure all their dependencies are also pulled.
k

Kevin Baker

11/07/2021, 2:47 PM
Hello @Frank and @thdxr. I’m starting to implement a data access layer and seeing this thread is super helpful. So you are using Kysely. Just curious have you looked into TypeORM? According to @revmischa it seems to work well with Serverless. While I’ve used it in the past, my concern on TypeORM is that it seems a bit bloated.
t

thdxr

11/07/2021, 3:29 PM
I personally prefer query builders to ORMs which is why I lean towards Kysely. Comes down to personal preference though - if you feel TypeORM is too heavy you probably will enjoy Kysely
k

Kevin Baker

11/07/2021, 3:46 PM
I used Knex a good but a while back too. I’ll have a closer look. One remaining concern is the light weight documentation for Kysely… do you think the work you are doing will lead into some code examples with SST/Kysely
t

thdxr

11/07/2021, 5:34 PM
Yeah we definitely will have more examples. Kysely is technically well documented
they heavily use jsdoc so you'll see docs right in your editor
p

Patrick Gold

11/07/2021, 5:37 PM
I've been using Kysely this week and it's been great. The inline jsdocs are super helpful with even examples showing up in vscode.
k

Kevin Baker

11/09/2021, 2:06 AM
Hey guys I think I almost have this going… should this work with MySQL in addition to Postgres? I’m getting the following error which seems to indicate the wrong driver is being selected…
ERROR BadRequestException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"tblCounter"' at line 1; Error code: 1064; SQLState: 42000
Ok , looking at
import { Driver, PostgresAdapter } from "kysely";
https://github.com/serverless-stack/kysely-data-api/blob/67face7b80e152d8bc2cfeeb1950c91bf003a7a3/src/data-api-dialect.ts#L1 It looks like it is bound to
PostgresAdapter
, but given that is the only reference to PostgresAdapter, I’m hoping I can possibly fork to a new
data-api-mysql-dialect
swap for
MysqlAdapter
.
If this is the case would it make sense to be more explicate about the dialect? So the current one would be
data-api-postgres-dialect
? and the “mysql” one would be
data-api-mysql-dialect
. Could also just do a single
data-api-dialect
and export both so we could use:
import {DataApiPostgresDialect, DataApiMysqlDialect } from 'kysely-data-api';
and keep it in one module.
I guess it comes down to the approach: 1) fork current data-api-dialect to make a new “MySQL Dialect” for atomic module. This makes some sense for long term extensibility for other dialects,.. Insert database here. 2) extend current dialect with multiple exported modules
DataApiPostgresDialect
and
DataApiMysqlDialect
to keep it all together. I suppose this thread might belong on the Kysely project.
t

thdxr

11/09/2021, 3:07 AM
I'm going to update it to support both. I was waiting for MySQL support to land which it has now
k

Kevin Baker

11/10/2021, 10:15 PM
In reading the documentation from AWS it seems that the Aurora Serverless V1 is maybe not meant for production... • Infrequently-used applications • Development and test databases I'm hoping for v2 in the near future with Data API, which seems to be better production fit based on the docs... that said for now do you guys free V1 is accessible for production applications using Lambda/SST? I specifically have needed MySQL for Geospatial queries for our application, but not kinda thinking maybe I should refactor this code to DynamoDB.
t

thdxr

11/10/2021, 10:25 PM
If you're using mysql have you considered planetscale?
Otherwise I do feel Serverless v1 can be used in production - you just can't set it to scale to 0
k

Kevin Baker

11/10/2021, 10:38 PM
Thanks Dax, I’m having to stick to AWS for this project so have really looked into Planetscale much I’m also super interested in the whole approach of Aurora Serverless.
BTW I forked your project based to build a MySQL specific Dialect, but paused. I think your right it makes sense to keep it in one package.
Let me know when your are ready with the Kysely Data API for MySQL I can test.
t

thdxr

11/10/2021, 10:43 PM
Let me play around with it tonight
k

Kevin Baker

11/10/2021, 10:44 PM
Cool, thanks!
t

thdxr

11/10/2021, 11:14 PM
got it working - that was easy 😄
I'm going to publish a new version - I didn't test it exhaustively so lmk if something is weird
pushed, it's version 0.0.7 and I started the basis of a Readme: https://github.com/serverless-stack/kysely-data-api
k

Kevin Baker

11/10/2021, 11:36 PM
I’ll dig in tonight Excited!!
Works! Thanks so much Dax! Awesome
t

thdxr

11/11/2021, 2:17 AM
cool!
7 Views