What's a good approach for managing changes to dynamodb tables? I find that it doesn't take much, and I end up in a complete mess. Then I just tear it all down and put it up again. It feels quite cumbersome in development, and it's not going to fly at all for production. In an SQL world, I'd have a starting point, then run migrations from there and use a framework for managing the migrations. Is there any kind of equivalent for Dynamo that integrates nicely? How do people manage the delta between the tables, keys and indexes defined in the stack, and the current state of the deployed dynamo tables? Everything else I think I've been able to live with quite happy, but dynamos lack of flexibility during development I find crippling. At least when I use CDK/SST it seems like let me change more than 1 GSI at a time. Just for some context, I'm working with a GraphQL, schema first approach with a 1:1 mapping to dynamo tables. But making some changes to the schema is painful 😕

Generally for manipulating global secondary indexes you shouldn't experience difficulties when doing it through stack code

I've seen the occasional article where people have paired up DDB with GQL but, whilst I don't have any direct experience, the two technologies don't feel like a good fit to me. One of the main strengths of GQL being the ability to define your access pattern on the fly. One of the main limitations in DDB (as a trade-off for speed) is the need to define all your access patterns upfront. Definitely watching this thread for other replies though, as curious if people are making this work

For dealing with changes to the shape of your data you'll need to write scripts that scan and rewrite your data. You can get pretty advanced with this and write step functions to parallelize this. The excellent OneTable library you should probably use also has some migration support. Additionally I would really avoid making a 1:1 mapping between your API layer (gql) and your database layer. You should build a middle domain layer that exposes useful functions that gql calls into and is totally unaware of your dynamo schema. It will let you reconfigure things later with minimal impact to your codebase

@thdxr CDK will only allow you to change one GSI at a time. We found also that often that change requires the GSI to be renamed as it can't be updated. So we had to remove the old GSI in one deploy, then add the new in a second deploy

Ah yeah was about to caveat that renames are weird in cloudformation

That's probably because we keep pestering you with questions about it 😄

Regarding GSI, you might want to consider generic GSI like

gsi1pk

&

gsi1sk

, etc. How does it work in two sentences ? You must consider that your indexes (primary and secondary) are not attributes of your data but that instead you expose parts of your data through your indexes. Then, when you create a schema or update your schema (e.g. migration), you only new to update (1) the data attributes themselves (data model) and (2) the data-to-index mapping. +1 for OneTable and their take on migrations using

.up(version="latest")

and `.down(version="rollback")`@thdxr

After a few of these issues, we are strongly considering simply managing databases in a separate stack. I've even seen data layers separated by accounts.

Thanks everyone. Going to check out OneTable, sounds like it might help with the migration stuff. I don't mind single table design, but it gets super confusing for new devs and feels really low level whenever you need to manually work with the data. And most of the benefits with provisioning that you used to get aren't as big an issue anymore. I found the 1:1 mapping works pretty well with GraphQL. We keep queries pretty simple. It's just the dev pain and random constraints that are brutal (like the cloud formation GSI changes). Feels like it's always working against you and the opposite of Agile...

Yeah OneTable deserves to be more popular (cc @Frank maybe we make it part of our new getting started guide) No sst integration with it right now but you can use it in combination with sst.Script

Just to provide a non-OneTable option, I just maintain a

TYPE

and

VERSION

field on every entity. A migration is just a table scan looking for certain values in these fields and acting accordingly. This part of DDB actually seems easier to me than SQL (unless you mean only adding indexes, which is definitely harder, but still not bad, and fully capable of no downtime). I also highly, highly recommend Alex DeBrie's book.

@Derek Kershner Absolutely agree on the

__type__

and

__version__

fields !