https://serverless-stack.com/ logo
#help
Title
# help
g

Garret Harp

12/07/2021, 3:39 PM
Is AWS having issues for anyone else? Seems everything died
t

thdxr

12/07/2021, 3:40 PM
good for me
"everything died" sounds bad!
g

Garret Harp

12/07/2021, 3:41 PM
Every lambda request was getting 500 errors seems to be working now (even seed.run was not working for like 5 minutes for me)
and aws sso is still down for me right now
m

Mischa Spiegelmock

12/07/2021, 3:42 PM
I’m getting 500s too now
us-east-1
g

Garret Harp

12/07/2021, 3:42 PM
yeah us-east-1 for me as well
m

Mischa Spiegelmock

12/07/2021, 3:42 PM
not seeing anything on https://status.aws.amazon.com/
t

thdxr

12/07/2021, 3:43 PM
ah I'm in us-east-2
yeah seeing it on twitter
g

Garret Harp

12/07/2021, 3:48 PM
It's back up and it looks like they might have just done an update that broke everything because the SSO login screen changed now lol
t

thdxr

12/07/2021, 3:49 PM
It looks the same in us-east-2
d

Devin

12/07/2021, 3:51 PM
there was also chatter in serverless about everything going down
s

Sam Hulick

12/07/2021, 4:36 PM
AWS status page says everything is fine though 🤦‍♂️
a

Austin

12/07/2021, 5:03 PM
DynamoDB seems to be the only service APIs that aren’t working for us besides the management console. Perhaps DDB is the root cause. Anyone else having issues with apps using DDB in us-east-1?
g

Garret Harp

12/07/2021, 5:05 PM
Dynamo is partially working for me - normal put/update/get/queries work but streams are not
or if the streams are working then its eventbridge not working because we push from streams -> eventbridge to actually handle events
a

Austin

12/07/2021, 5:07 PM
Yeah, it is weird. Scans are working for me but some query operations aren’t.. but the requests work intermittently
g

Garret Harp

12/07/2021, 5:10 PM
assuming issues are still loading from cloudwatch on seed then I havent had a single operation fail from dynamo
Oh they updated status again apparently dynamo is having issues says "control plane apis" not really sure what that means tbh
g

gio

12/07/2021, 5:12 PM
what normal people do without aws? Should I go outside? Damn it’s cold
s

Sam Hulick

12/07/2021, 7:04 PM
holy crap, this is a serious outage.. geez
t

thdxr

12/07/2021, 7:04 PM
yeah I'm kinda stuck
s

Sam Hulick

12/07/2021, 7:05 PM
I’m glad the most critical part of our application isn’t down (it’s old school, hosted on EC2). our customers would be suuuuper pissed
g

Garret Harp

12/07/2021, 7:06 PM
Only thing I have noticed is either dynamo streams or event bridge... not sure which is the one having issues since I cannot login to the console to see which lambdas are running 😅
Lambdas were only down for a few minutes at one point
s

Sam Hulick

12/07/2021, 7:07 PM
some of my API endpoints work, some don’t. basic Lambda funcs are OK.. but one of my API calls that gets a CloudFront signed cookie fails completely
t

thdxr

12/07/2021, 7:18 PM
Anything that requires on "global" infrastructure is down
So cloudfront for example is global, and all the stuff that runs globally depends on us-east-1 for their control plane
s

Sam Hulick

12/07/2021, 7:19 PM
ahh
t

thdxr

12/07/2021, 7:19 PM
that's why I avoid us-east-1 - it's where aws runs a bunch of their own stuff and is burdened in general
s

Sam Hulick

12/07/2021, 7:20 PM
I was torn between choosing us-east-1 and us-east-2. I figured us-east-1 is a better option, because if there’s an outage in that region, it affects a larger part of the Internet. so if we’re down, people will just see it as “the Internet is broken”
whereas if we’re on a less populated region and we’re down, our customers will think it’s just us
t

thdxr

12/07/2021, 7:23 PM
yeah that's true
tradeoffs
s

Sam Hulick

12/07/2021, 7:24 PM
the big question is, why is Amazon.com working perfectly? 🤔
t

thdxr

12/07/2021, 7:25 PM
apparently it's not, my friend was saying he had some issues checking out his cart
s

Sam Hulick

12/07/2021, 7:26 PM
ahhh. I was just browsing around, looking at my orders, etc. all totally snappy
2 Views