https://serverless-stack.com/ logo
#help
Title
# help
d

Daniel Gato

04/12/2022, 11:59 AM
Apparently, I’m stuck on
Copy code
deploy stack: run cdk deploy
deploy stack: poll stack status
deploy stack: poll stack status: {
  StackStatus: 'UPDATE_COMPLETE',
  LastUpdatedTime: 2022-04-12T08:11:52.538Z
}
deploy stack: poll stack status: unknown
deploy stack: poll stack status: {
  StackStatus: 'UPDATE_COMPLETE',
  LastUpdatedTime: 2022-04-12T08:11:52.538Z
}
deploy stack: poll stack status: unknown
....
We are 2 people experiencing the same on the same project I already tried to delete .sst and .build just in case. Edit: it worked for me but my colleague is still stuck on that for now 3h
f

Frank

04/12/2022, 2:28 PM
Hey @Daniel Gato, if u go to ur AWS CloudFormation console > go into the stack > go to the Change sets. Do you see anything in the list? If so, what’s its status for ur colleague?
d

Daniel Gato

04/12/2022, 2:53 PM
My colleague had only 2 stacks out of 8
Mine worked at the end
f

Frank

04/12/2022, 2:56 PM
can u check if any of those 2 stacks have a change set?
d

Daniel Gato

04/13/2022, 10:07 AM
Apparently, it just takes way more time than before
it went trough after 4 or 5 hours
a

Athol Birtley

05/06/2022, 1:49 AM
Hi @Frank - we are experiencing the same problem after upgrading from 0.69.3 to 1.0.0-beta.21 following the upgrade instructions here . We are getting the following terminal output on loop:
Copy code
deploy stack: poll stack status: unknown
deploy stack: poll stack status: {
  StackStatus: 'UPDATE_COMPLETE',
  LastUpdatedTime: 2022-05-05T09:08:09.531Z
}
When viewing the stack in Cloudformation Management Console, there are no change sets (but there is a ‘last executed changeset’ from 2022-05-05 T090809). I will leave it running to see if eventually goes through, like Daniel did above, but thought you might like to know / might have an alternative suggestion.
I’ve also just upgraded to SST 1.0.2 (CDK: 2.20.0), deleted .build and .sst directories, and tried again, and the same issue occurs.
Update: the change set appeared in the management console after approximately 28 minutes, and is now showing as CREATE_COMPLETE. SST then spent about 10 minutes on “Fetching stack events”, and was then ready to accept live lambda requests. FYI, I don’t think the stack itself is very big - it has 1 EC2 instance, 1 VPC, 1 Security group, 10 API endpoints, 1 standalone function, a static site, and an EventBus with 2 rules (and a function for each rule). Now that it has “taken its time” once,
sst start
is very quick to get to “listening for requests…”
To give extra context - I am using the
ap-southeast-2
region. Also, I just tried creating a brand new SST project, with only 1 API endpoint, and that was also incredibly slow to
sst start
. Took at least 15 minutes for the debug stack to even appear in AWS (but it then created quite quickly), and then a further 12 minutes for the MyStack to appear in AWS (although again, once it appeared in completed quite quickly). It took 5 more minutes for SST to realise that the MyStack stack had finished creating though (perhaps due to an exponential backoff in polling?)
Update: Making tiny changes after the initial cloudformation stack creation has succeeded is not always fast. Even with only 1 API endpoint in the stack, a trivial update can take north of 20 minutes. (And most of this time is waiting for the stack update to appear in AWS, then waiting for SST to detect that the update has finished.)
Update: It seems that in addition to taking ages to appear in AWS, stack creations can linger in a
REVIEW_IN_PROGRESS
state for a very long time (eg 25 minutes), before they actually start doing any work in Cloudformation. I’m not overly familiar with cloudformation / CDK, but is this because the change set is being created but not being manually ‘executed’ ?
I’m finding that SST is basically unusable following the upgrade to v1. Waiting 20+ minutes for any stack change to begin (let alone finish) is a real productivity killer 😅 Suggestions would be most welcome, as I’d prefer not to downgrade to an earlier version and be stranded on an unsupported alpha product 😄
(The 20+ minute wait also applies if there are no changes. Eg, you stop debugging in VS Code, then start debugging again, with zero changes made to the stack. Takes ages just to verify that the debug stack is unchanged, let alone process the MyStack)
I’m not sure if this is an issue with my SST configuration or project setup, or a broader issue with CDK/AWS, or even an AWS partial outage that isn’t being reported on the AWS status page, but I have created a reproduction for issue 1559 which also (on my machine) reliably reproduces this lengthy delay when running
sst start
.
l

Lewis Eccles

05/06/2022, 2:50 PM
I was about to post about how the new instructions for creating an SST app, after the v1.0 release is being incredibly slow for me. SST start, SST remove are taking roughly 10 mins when previously they'd take only up to a couple of mins. I'm literally deploying APIG with a single endpoint right now and its taking 10 mins
a

Athol Birtley

05/07/2022, 9:07 PM
Looking into this a bit further, when I upgraded to SST v1, I had to upgrade my CDK dependency from 2.15 to 2.20 (as I am creating EC2 instances with no native SST construct(?)) in order to match SST’s new version 2.20. (Not relevant, but CDK changed some types eg from Vpn to IVpn). I went digging through the CDK release history, and saw that CDK release 2.21.0 included a fix relating to hangs. I thought that might be relevant. So, I bumped my CDK dependency version 2.21.0 and pinned it using the
"resolutions"
parameter in my package.json to ensure SST was using it too. I also deleted .build and .sst. Running
sst start
displayed a warning about mismatched versions of AWS CDK packages, which was expected. However, it did not solve the problem of take ages to detect whether the debug stack needed updating or not. (I stopped it after 10 minutes as I figured it wasn’t worth waiting any longer.) Then, I reverted my project to use SST v0.69.3 (what I had been using previously, when things were working fine), and reverted CDK to 2.15.0. Deleted .sst and .build again, and ran
sst start
. Still no luck! (Minutes and minutes just to determine whether the debug stack was deployed (which it was)). Then I gave up for the day, and tried again a day later. And what do you know - everything worked! Both SST 1.0.2 / CDK 2.20, and SST 0.69.3 / CDK 2.15.0 behaved as I was used to. It does take a minute or two to deploy when you switch from pre-1.0 to 1.0, but then if you stop
sst start
and run it again, it is very quick to detect no changes to the stack and start listening for live lambda requests. My thinking is that it was either a poor-connection issue at my end (that wasn’t resolved by upgrading CDK to 2.21.0), or that AWS Cloudfront in ap-southeast-2 was just having a really bad day. I’m leaning towards the latter, since I wasn’t having issues with anything else on the internet that day. Hope this helps!
Further update after a few more days of experimentation - the problem persists when connected to the internet on Wifi, but not when tethering on a 4G mobile phone, so it sounds like I might need to get the office router rebooted. But hopefully this helps the next person who stumbles upon the issue.
After restarting the modem, the problem has gone away. It’s strange, because the rest of my internet experience was fine (email, websites, video calls, etc). I guess it’s possible that AWS was throttling the old IP address, and by getting a new IP via the modem restart, we are now avoiding that throttle? Or it may just be a dodgy modem / ISP…
2 Views