https://serverless-stack.com/ logo
Join Slack
Powered by
anyone know of the best way to address this with l...
# help
s
anyone know of the best way to address this with little to no down-time? (now that we’re in production) this is a result of making the change to the 
HttpUserPoolAuthorizer
syntax
Copy code
dev-microservices-api-base | UPDATE_ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | dev-microservices-api-base | Export dev-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94 cannot be deleted as it is in use by dev-microservices-api-files, dev-microservices-api-media and dev-microservices-api-misc (and 2 more)
at a glance, it seems like I’d have to temporarily comment out all routes. except maybe the public ones
ugh. this is tricky and it’s a bit beyond me. @Frank @thdxr any help would be greatly appreciated. I’m happy to pay for your time.
t
Can you try doing sst deploy --stack dev-microservices-api-files
Not just that one, deploy all the problematic ones one by one
s
ahh.. so forcibly deploy the stacks that are complaining about being dependent first
t
or you can create a fake output with that same name in the first stack
Yeah it's dumb cfn doesn't handle this automatically
s
yeah, this is one of the worst parts of CFN/CDK. I wish it were smart enough to make changes in reverse order
gimme a bit. since I’m deploying, it’s gonna want to rewire all the debug hooks
Copy code
❌  dev-microservices-api-files failed: No export named dev-microservices-api-base:ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766 found
so this is tricky, because you can’t use 
this.exportValue()
on the 
HttpUserPoolAuthorizer
hang on. lemme just do 
yarn sst deploy
first.
Copy code
❌  dev-microservices-api-reels failed: Export dev-microservices-api-reels:ExportsOutputRefRouteGETreelsreelIdpresentationHttpIntegration860f00925152f0afcca0fe4a6d60923aE2A73EAFC1839436 cannot be deleted as it is in use by dev-microservices-api-public
man
so.. I think if I just keep following the trail of “in use by” [stack] and deploy that stack only, and work my way backwards, it should fix everything?
t
yeah exactly
s
@thdxr THANK YOU 🙌
t
understanding this is the final step to being a cloudformation master
congrats
s
haha
I’m embarrassed to say I forgot about the 
--stack
option. I’ll remember that next time, that’s incredibly handy
so, uh.. I’m repeating this in production now, and it’s not going so well 😕 same error as before:
Copy code
❌  prod-microservices-api-base failed: Export prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94 cannot be deleted as it is in use by prod-microservices-api-files, prod-microservices-api-media and prod-microservices-api-misc (and 2 more)
but this time when I try to deploy one of those individual stacks, it fails too:
Copy code
❌  prod-microservices-api-files failed: No export named prod-microservices-api-base:ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766 found
and
Copy code
prod-microservices-api-misc | UPDATE_ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | prod-microservices-api-misc | No export named prod-microservices-api-base:ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766 found
do I just have to keep going through every stack until one of them works?
t
The other option is to add an output with that name to the stack that it was removed from
ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766 I think, can put a dummy value
s
ok, so.. in api-base, 
this.exportValue('dummy', {name: 'ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766'})
?
t
Yeah I think that will work, @Frank is more familiar with this trick. Let me see if I can find it in slack
s
yeah, in this case it’s an 
HttpUserPoolAuthorizer
which can’t be exported. so hopefully the dummy trick works
step 2 is rather frightening in production. removing the reference to the authorizer would kill our entire app
t
We've been talking about hacking CFN to use SSM instead of CFN exports so the coupling is looser and they can be deployed out of order. But creates other issues where things might be broken during the deploy
f
Sam, if you go to CFN console and find this stack
Can you see what’s the value for 
ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766
?
s
in api-base? it doesn’t exist. instead, it’s 
ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94
f
oh i might’ve missed some context, did u get the 
No export named XXXX
when you tried to deploy a specific stack using 
--stack
?
s
sorry. so yeah, when I deploy all stacks, I get:
Copy code
prod-microservices-api-base | UPDATE_ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | prod-microservices-api-base | Export prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94 cannot be deleted as it is in use by prod-microservices-api-files, prod-microservices-api-media and prod-microservices-api-misc (and 2 more)
the value of that is 
prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94
f
can u go to the 
prod-microservices-api-base
stack and see what the value is for 
ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94
?
s
yeah, that’s the value above
lemme fix the name of that in my 
exportValue()
and redeploy
er wait
f
that’s the export name… can u take a screenshot?
s
sorry. the value is 
mb7pmm
ok, dumb question: if I exportValue a dummy using the name 
ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94
in api-base, do I deploy just api-base? or one of the stacks that depends on it?
ok, so with the dummy export, when I deploy just api-files, I get:
Copy code
prod-microservices-api-files | UPDATE_ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | prod-microservices-api-files | No export named prod-microservices-api-base:ExportsOutputRefRestApiHttpUserPoolAuthorizerF0D1F4E162D79766 found
f
let’s do a quick workaround, add this to the api-base stack:
Copy code
this.addOutputs({
  TempOutput: {
    value: "mb7pmm",
    exportName: "ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94",
  },
});
this creates a placeholder export
s
ok. and then deploy api-base?
f
with this, you can try deploy all stacks
s
ah ok
trying now
I’ll have to add this to our repo’s README as it’s likely to be a gotcha in the future
Copy code
prod-microservices-api-base | UPDATE_ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | prod-microservices-api-base | Export prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94 cannot be deleted as it is in use by prod-microservices-api-files, prod-microservices-api-media and prod-microservices-api-misc (and 2 more)
f
Sorry my bad, try this
Copy code
this.addOutputs({
  TempOutput: {
    value: "ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94",
    exportName: "prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94",
  },
});
s
the value wouldn’t be 
mb7pmm
?
hang on.. this looks promising
I’m seeing more green than yellow 😄
f
OMG.. sorry.. i’m stupid
s
haha it’s ok. it’s actually working
oh wait
spoke too soon
f
Copy code
this.addOutputs({
  ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94: {
    value: "mb7pmm",
    exportName: "prod-microservices-api-base:ExportsOutputRefRestApiUserPoolAuthorizer34D6B3E8DA3B1D94",
  },
});
s
ahhhh right
I was wondering where the key value went
ok, let’s try again
ah! a new error.
Copy code
prod-microservices-api-base | DELETE_FAILED | AWS::ApiGatewayV2::Authorizer | RestApiUserPoolAuthorizer34D6B3E8 | Cannot delete authorizer 'UserPoolAuthorizer', is referenced in route: POST /templates (Service: AmazonApiGatewayV2; Status Code: 409; Error Code: ConflictException; Request ID: 780a77e6-50bc-4518-9254-c346fcffd62a; Proxy: null)
however, there were some 
CREATE_COMPLETE
lines in api-base with regards to the http authorizer
f
Oh can you show me your authorizer code?
s
so I’m wondering if I can remove the dummy output now
f
probably not, i think other stacks are still depending on it
we can remove this LAST after everything works
s
Copy code
this.restApi = new sst.Api(this, 'RestApi', {
      accessLog: {
        retention: 'ONE_MONTH',
      },
      defaultAuthorizationType: sst.ApiAuthorizationType.JWT,
      defaultAuthorizer: new HttpUserPoolAuthorizer(
        'HttpUserPoolAuthorizer',
        props.cognitoAuth.cognitoUserPool!,
        {
          userPoolClients: [props.cognitoAuth.cognitoUserPoolClient!],
        }
      ),
nothing too fancy with the authorizer. it’s just that I had to change the syntax due to CDK v2
f
Do u remember what it looked like before u changed it?
s
Copy code
defaultAuthorizer: new HttpUserPoolAuthorizer({
        userPool: props.cognitoAuth.cognitoUserPool!,
        userPoolClients: [props.cognitoAuth.cognitoUserPoolClient!],
      }),
f
in the new code, can you change
Copy code
new HttpUserPoolAuthorizer("HttpUserPoolAuthorizer", ...
to
Copy code
new HttpUserPoolAuthorizer("Authorizer", ...
s
it had no name, so it looks like it made one up by concatenating the construct name + a generic authorizer suffix. 
RestApiUserPoolAuthorizer
sure, and then deploy just api-base?
f
And comment out the 
this.exrportValue
line.
can u just run 
sst build
and send me the template for 
api-base
?
s
sure thing
thanks a billion for your help!
f
I have a feeling the issue is due to the name of the Authorizer has changed
s
yeah, most likely.
f
leading to export name changed
CFN is trying to remove and recreate the authorizer
s
yep
so I should name it 
Authorizer
, not 
RestApiUserPoolAuthorizer
? (that was the old name)
f
yeah name it 
Authorizer
s
hmm, someth ing new is happening. all the routes are updating now
it’s deleting & recreating all the routes. and I didn’t do your change yet.
f
wait did u run 
sst deploy?
s
well, I hadn’t run the build command yet. it was still deploying from last time
so I was letting it finish
f
ah i see yup
s
now it looks promising
it deployed!
now I’m not sure what to name it 😄 I’ll try Authorizer, but it may try to delete it again
now there are two authorizers.. hmmmm
oh, one is probably the dummy output
f
ah if it worked, don’t have to change the name
did u deploy just the api-base stack or all stacks?
s
yeah that was a full deploy.
f
oh nice.. u can just remove the dummy 
exportValue
we put in.
s
whew. ok.. all is well and back to normal! talk about a breaking change 😅 thanks again, @Frank & @thdxr. @Jay I owe you a call (I hadn’t forgotten). let’s set up a time. I owe you guys one
f
Haha glad it’s working! 👍
s
whoever on the CFN team decided that a rename should delete a resource needs a talking-to 😄
what an absolute pain
j
No worries! Glad we could help!
Open in Slack
PreviousNext
Powered by