https://serverless-stack.com/ logo
Join Slack
Powered by
welp, that’s a new error from CloudFormation: `Rec...
# random
s
welp, that’s a new error from CloudFormation: 
Received response status [FAILED] from custom resource. Message returned: Rate exceeded
😄
a
You’re DDOSing AWS. 😂🙏
s
AWS is DDOS’ing itself! 😄
AWS seems a bit stumped. but it looks like it’s probably something to do with all those 
Custom::LogRetention
resources
@Frank @Jay is there any reason you didn’t just have SST create log groups via 
AWS::Logs::LogGroup
instead of creating a bunch of custom resources to set the log retention?
or this a CDK thing
YEP
I went with this, to be super safe:
Copy code
logRetentionRetryOptions: {
      base: Duration.millis(500),
      maxRetries: 15,
    },
@thdxr @Frank I’d suggest maybe baking this into SST as a default, so no one has to suffer like I did 😄 and waste hours trying to figure it out
I can’t see any advantage to the defaults of 100ms and 3 retries. especially if it doesn’t do the trick
f
@Sam Hulick yeah CDK doesn’t create the log groups for Lambda functions in CloudFormation. Lambda functions will automatically create the log group when they get first get invoked.
So if you are configuring the log retention, CDK has to do that in a custom resource.
Btw, how many functions did you add?
s
will check in a sec. is there some quick way to tell? my code is pretty spread out
oh I can just look at the # of lines for “Building Lambda func..” in the CLI
I’ll let you know in a bit
f
Oh are you deploying ur app to a fresh account?
s
55 functions. yeah, deploying to stage prod in my primary account now
I got lucky in the other accounts for some reason.. didn’t hit that retry rate limiting. it’s just pure chance I guess
sorry, 42 functions in the API stack, actually. 55 across all stacks
f
ah I see.. that makes sense
s
thanks!
what an obscure issue to run into 🙂
t
Woah I think I just got this too for the first time ever
I'm not even doing anything crazy
Copy code
Received response status [FAILED] from custom resource. Message returned: Waiter InvalidationCompleted failed: Max attempts exceeded (RequestId: ded6e1ea-8b11-44ae-ae1e-4a4b8d24c1a3)
s
damn it, AWS! 😄
t
This was on deploying a StaticSite which has a CRD for replacing values
s
let me know if the log retention retry values above work for you
if it’s a log retention resource that failed, of course
f
@thdxr the custom resource triggers a CF invalidation and waits for invalidation to finish. I think that’s causing the error.
t
Yeah it worked on the second try
f
I see
t
I noticed this was in python the other day, was there a specific reason for it?
f
b/c I want to show off my python skillz
the code is based on CDK’s custom resource in 
@aws-cdk/aws-s3-deployment
And we rewrote some of it to get around 500MB Lambda disk space, better S3 cache policy, and made it faster
t
Ah ok cool
s
thanks for your help today, guys 💪 I managed to get our new SST back end deployed to prod in our primary account
j
That’s awesome!
3 Views
Open in Slack
PreviousNext
Powered by