We re seeing the error ```Replacement type updates not suppo Serverless Stack #sst

We're seeing the error: ```Replacement type update...

Ross Coundon

11/02/2021, 4:45 PM

We're seeing the error:

Copy code

Replacement type updates not supported on stack with disable-rollback.

We had to rename a table in the stack as its definition had changed, after doing so we get this error. What's the recommendation to deal with this?

Frank

11/02/2021, 4:46 PM

Let me loop in @thdxr . Dax, I think this might be related to the disable rollback change for sst start.

thdxr

11/02/2021, 4:47 PM

You can pass in a flag to SST start

--rollback=true

Frank

11/02/2021, 4:47 PM

When rollback is disabled, u can't replace resources in the stack. (Updating resource names often require a replace ie. delete and recreate)

Ross Coundon

11/02/2021, 4:48 PM

cool, I'll give that a tru

Ross Coundon

11/02/2021, 4:48 PM

try even

thdxr

11/02/2021, 4:48 PM

We disable by default because it's faster and 99% of the time it's fine

Ross Coundon

11/02/2021, 4:52 PM

got ya

Ross Coundon

11/02/2021, 4:54 PM

Lost our window to try this out so trying redeploy from seed and getting

Copy code

This stack is currently in a non-terminal [UPDATE_FAILED] state. To update the stack from this state, please use the disable-rollback parameter with update-stack API. To rollback to the last known good state, use the rollback-stack API

I don't see the option in the console to apply a rollback, which seems odd. Is there a way to force this via seed

Ross Coundon

11/02/2021, 4:54 PM

Frank

11/02/2021, 5:13 PM

Ross, if you go to the CFN console, can you rollback manually there?

Mehmet Ali SARAÇ

11/02/2021, 5:17 PM

is this error same as this ?

Copy code

CloudFormation cannot update a stack when a custom-named resource requires replacing. Rename *********** and update the stack again.

Mehmet Ali SARAÇ

11/02/2021, 5:18 PM

@Frank I'm trying to add new globalIndex on dynamodb table

Ross Coundon

11/02/2021, 5:19 PM

No - there's no option to rollback

Ross Coundon

11/02/2021, 5:20 PM

Previously I've seen the banner appear at the top with the options to rollback, update, etc but nothing's there this time

Frank

11/02/2021, 5:23 PM

Can you try reverting the change and deploy again from local without the —rollback=true flag?

Ross Coundon

11/02/2021, 5:23 PM

Great minds, actually doing just that but with rollback=false

Frank

11/02/2021, 5:26 PM

@Mehmet Ali SARAÇ that might be a different issue. Were you trying to rename a global index? You’d have to remove it, deploy, add it back with the new name and then deploy again.

Mehmet Ali SARAÇ

11/02/2021, 5:27 PM

nope I didnt change the name of it. I just simply add a new global index

Mehmet Ali SARAÇ

11/02/2021, 5:27 PM

how could I restore the data ? @Frank

thdxr

11/02/2021, 5:29 PM

Are you sure you're adding a global index and not a local index?

Mehmet Ali SARAÇ

11/02/2021, 5:30 PM

Copy code

globalIndexes: {
                isApprovedIndex: {partitionKey: "is_approved", sortKey: "timestamp"},
                fromIndex: {partitionKey: "from"} // this one is new
            }

Mehmet Ali SARAÇ

11/02/2021, 5:30 PM

yep I did this too many times

Ross Coundon

11/02/2021, 5:31 PM

@Frank - this gave me

Copy code

Replacement type updates not supported on stack with disable-rollback.

Mehmet Ali SARAÇ

11/02/2021, 5:31 PM

But this error happens after an update which secondaryIndexes deprecated

thdxr

11/02/2021, 5:34 PM

@Ross Coundon I think you'd need to revert to the old code, then run with rollback=true

thdxr

11/02/2021, 5:34 PM

wait actually, you can't fix this from the CFN console? I remember being stuck in this state and using that

Mehmet Ali SARAÇ

11/02/2021, 5:36 PM

aha after I change the name of the dynamodb table I got the error which @Ross Coundon gets

Copy code

Replacement type updates not supported on stack with disable-rollback.

Ross Coundon

11/02/2021, 5:36 PM

It's not giving me the option to perform any changes via the console

thdxr

11/02/2021, 5:43 PM

There's some combination of enabling/disabling rollback + reverting code changes in a certain order that will get it back. I'm not able to remember exactly what I did though

Ross Coundon

11/02/2021, 5:45 PM

Nuts - seems I'm stuck in a catch-22 Reverting to the old code and running deploy with

--rolback-true

gives

Copy code

This stack is currently in a non-terminal [UPDATE_FAILED] state. To update the stack from this state, please use the disable-rollback parameter with update-stack API. To rollback to the last known good state, use the rollback-stack API

Ross Coundon

11/02/2021, 5:45 PM

I have to jump in the car for an hour now, will be back to fix this later

Ross Coundon

11/02/2021, 5:46 PM

any ideas in the meantime, gratefully received!

thdxr

11/02/2021, 5:47 PM

old code + rollback=false didn't help either?

Frank

11/02/2021, 6:53 PM

@Ross Coundon @Mehmet Ali SARAÇ Dax and I are going to jump into a meeting for 2 hrs. We can try to reproduce the issue on our end after that. If you guys figure anything out before that, keep us posted!

Mehmet Ali SARAÇ

11/02/2021, 7:05 PM

I rollback the stack and it worked.

Ross Coundon

11/02/2021, 7:14 PM

How did you roll it back?

Ross Coundon

11/02/2021, 7:33 PM

@thdxr - old code + rollback=false gives

Copy code

Replacement type updates not supported on stack with disable-rollback.

Ross Coundon

11/02/2021, 8:00 PM

I've now run the deploy with

rollback=false

and then

rollback=true

which both failed but at least then the console allowed me to initiate a rollback which succeeded and left the stack in state UPDATE_ROLLBACK_COMPLETE So, now I'm attempting to deploy the old code via Seed...

Ross Coundon

11/02/2021, 8:03 PM

Copy code

Status: Failed
  Error: Internal Failure

Ross Coundon

11/02/2021, 8:03 PM

From Seed build logs

Ross Coundon

11/02/2021, 8:08 PM

Just retried, same problem - 'internal failure'

thdxr

11/02/2021, 8:27 PM

is this not a local development environment? Curious why going through seed

Ross Coundon

11/02/2021, 8:30 PM

We had an app that was deployed via seed. We tried to switch it to run locally to debug some changes and got into this tangle

Ross Coundon

11/02/2021, 8:31 PM

Now we're just trying to get it back to the state it was originally

Ross Coundon

11/02/2021, 10:20 PM

So, I ended up running

sst remove

from my own machine followed by a seed deploy which (after removing log retention settings) worked.

Ross Coundon

11/02/2021, 11:07 PM

What would you recommend as a process to mske a change to something like the structure of a dynamodb table in terms of deployment? There might need to be some data migration in that example but what would need to be done from an SST and Seed perspective?

thdxr

11/02/2021, 11:49 PM

So I don't fully understand what change forced a replacement. Did you rename the literal table or the logical id?

Mike McCall

11/14/2021, 3:31 PM

I am having this issue. The problem with replacement types is that subsequent deployments do not detect changes. It essentially just ignores the update and moves on.

Mike McCall

11/14/2021, 3:45 PM

if you try with rollback after a fail you'll run into errors like:

Copy code

This stack is currently in a non-terminal [UPDATE_FAILED] state. To update the stack from this state, please use the disable-rollback parameter with update-stack API.

Mike McCall

11/14/2021, 3:45 PM

Not sure it's the best default

Mike McCall

11/14/2021, 3:47 PM

Basically, you need to remove, deploy, add, deploy. This is not good especially since my current issue is with an api authorizer.

Ross Coundon

11/14/2021, 4:03 PM

@thdxr I missed your question when we were talking about this. For us it was the definition of the global secondary index had changed

Frank

11/15/2021, 6:23 PM

@Mike McCall

… subsequent deployments do not detect changes.

I think that might be a bug with SST. I’m going to put in a change today https://github.com/serverless-stack/serverless-stack/issues/1037

Frank

11/18/2021, 9:47 AM

This is fixed in v0.53.0.

Joe Kendal

11/25/2021, 11:33 AM

@Frank upgraded to v0.53.0 and have the same issue. Changed a lambda function not a table

Frank

12/01/2021, 6:07 PM

Hey guys, this is fixed in v0.53.3.

Frank

12/01/2021, 6:07 PM

sst start

uses

--rollback=true

by default now. And you can set it to

false

to override it.

Open in Slack

Previous Next