Now looking into serverless I'm wondering if I should go back to thinking in terms of simple k/v stores

SEED uses DynamoDB full on. We’ve been very happy with it. That’s 1 data point for u lol.

I am a big fan of Postgres, but once you get your head around single-table design it’s quite hard to go back to writing data migration and all of the other admin junk. I do kind of hate the vendor lock-in of dynamo though …

Yeah agreed. We’ve been using multi-table (1 table per object type). And the performance has been very good and reliable, so we haven’t invested time to look into single table.

for single-table dynamodb i’ve used this library and it’s really improved my code. in no small part because it improved my approach and the design of my code: https://github.com/jeremydaly/dynamodb-toolbox

We just migrated from dynamodb to postgres. Biggest pain points of dynamodb were migrations, hard to connect to any visualization tool, no joins…

That's helpful to hear. Been going in a circle on this one, most of my experience has been getting good with things like denormalizing in nosql databases. But as I started to use Postgres it was kind of nice not having to worry about accessing data in a variety of different ways

I think the biggest thing is really embracing single-table design, but it requires a LOT more planning up front. I really prefer not doing joins an getting back everything you need in one query a discreet items

Yeah I’ve been using dynamo as my primary store for a few years now, been happy with it - but it is annoying sometimes to have to worry about data design upfront. On the flip side…I know that I’ll get the query performance I’m getting now at a much higher throughput.

^ exactly the same situation. I got used to the patterns and it became less annoying

Also I think you’re “supposed” to use the Data API with postgres and serverless, but the Data API has/had some issues vs going directly. I’m just parroting from blog posts on the topic though, no first hand experience:

The data api does help a bunch of things since PG has a broken connection model

One more thing: At first I was stingy creating GSI's on my single table design because aws limits tables to 20 of them. Then I learned you can make a request to aws and they'll allow 100. So the lesson is not to be stingy with GSI's and embrace them instead. Although others may have different views on this and I'd be curious to hear their experience.

ah interesting, I'm used to not having secondary indexes at all as an option

we try to keep GSIs to <= 5, but we don’t actually care as long as we are getting our access patterns covered. We also re-use them so we give them generic names like GSI1PK/GSI1SK

How do you all deal with data integrity issues - for example ensuring there’s only every 1 user registered under a certain email

I literally just wrote code for this - except its an idempotency key instead of a unique email (using the excellent

dynamdb-toolbox

that David mentioned)

nice

@Matthew Purdon tone it down there! We need ease new initiates into the cult

lol

nah I'm already in hit me as hard as you want

What is the reason you try keep your GSI's <= 5, if you don't mind sharing? For me, I just know that when I start considering putting in two records for the same data so I can accommodate two access patterns, it could be time to consider a GSI. But again, I would love to hear what others are doing.

well, back in the day that was the limit for the number of gsis you could have. GSIs aren’t free so we try to use that rule of thumb to make us think more about what we can re-use for new access patterns, we also try to limit the fields in the GSI so it’s more efficient

yeah that’s true they aren’t free. re-use is a smart strategy, i haven’t tried it but i could definitely use that approach. another benefit is that you can only modify one index at a time during a deployment so you’d reduce likelihood of needing to create > 1 index during a big upgrade. i found that scenario to be a bit of a pain. the only concern i have is that could potentially cause confusion. has that ever been a problem?

@Matthew Purdon @thdxr - if you like that you should read The DynamoDB Book by Alex DeBrie, it’s a phenomenally good resource for single table design

yeah, I have (I linked his blog above) it’s really great as well.

I've been a fan of single table design in dynamodb, however with graphql its not as necessary (Alex Debrie even states this) and if you plan to use streams is very wasteful.