Hey everyone, I’m a frontend dev building my firs...
# help
j
Hey everyone, I’m a frontend dev building my first sst app, so my knowledge of this stuff is extremely limited. I received a concerning email from AWS this morning stating that I’ve exceeded 85% of the free tier usage limit for the
AWSQueueService
service (857,066 requests), and I have no idea where this is coming from. It looks like it’s coming from
SQS NumberOfEmtpyReceives
. Can anyone shed some light onto what’s causing this and how I can stop all these requests? This is without my app running. Apologies for the ignorance.
g
m
👆 how are you consuming messages? It sounds like your consumer(s) are not long polling. The wait time for short polling is 0 seconds.
Each consumer is a multiplier. If using lambda you can also set provisioned concurrency to limit the number of consumers for that trigger.
j
I'm not consuming messages directly. Seems like an internal sst thing.
I think it’s being triggered by a
SiteRegenerationFunction
lambda.
Is it safe to simply delete the queue to stop it from firing, or would that possibly mess up my build?
I don't have enough knowledge to know why this is continuously running in the background when my app isn't running. Seems like something’s messed up, and it's going to start incurring costs shortly.
s
Hey John, this is related to how AWS Lambda consumes messages from SQS. Lambda internally checks for new messages over and over, regardless of you have messages to process or not. I believe you can adjust the polling interval directly, but haven't done much with that myself.
Do you have an SQS queue in your application?
j
I haven't added one, which makes me think it's an sst thing, but not sure. I deleted the queue resource, but haven't tried running my app again to see if it'll cause any issues.
I don't know anything about SQS. Only learned of it today from the AWS warning email.
s
SST deploys a few things to enable the live lambda dev environment, but no queues as far as I know. Might be worth checking out what queues are in your account via the AWS console
j
Yeah I took a look and this was the only one for my current stage. Looked like it was being triggered by a
SiteRegenerationFunction
Lambda. Also looks like it was coming from my front end nextjs build.
t
Oh my god - I got the exact same thing
Was wondering what the cause was
I currently use the Topic, NextjsSite and Api construct
j
Glad I'm not the only one seeing this. Maybe @Frank or @thdxr have an idea? Looks like it might be part of NextjsSite.
m
Took a look and I did find SQS being used in the NextjsSite code. You were right @John it has to do with site regeneration. https://github.com/serverless-stack/serverless-stack/blob/8113c5c329809a716ae499eba7e725c9a7ad930f/packages/resources/src/NextjsSite.ts#L454
s
Hmm, that's interesting. Maybe it would be a good idea to have a section in the documentation that lists what resources are created when using the higher-level construct?
j
It can't be normal for a queue to have that many requests during development right? Seems like it was firing out of control.
m
Agree. Seems to be a bit much.
f
Hey guys, yeah
RegenerationQueue
is from the
NextjsSite
construct. Taking a look..
Just tried it out.. I can confirm I’m also seeing 15 Empty Receives per minute, same as what @John and @Tharshan saw. @John @Tharshan can you try going into ur SQS console, set the “Receive message wait time” to 20 seconds, and see if that brings down the empty receives count?
I tried setting “Receive message wait time” to 20 seconds, but that didn’t seem to affect the Empty Receives rate (still 15 per sec).
Any SQS experts know what’s going on? Btw, NextjsSite uses a FIFO queue.
s
AWS has a support center article about SQS empty receives and billing, along with some best practices to reduce SQS costs.
The most common cause for a high number of SQS requests are empty receives, which are billed as ReceiveMessage requests that don’t return a message. Amazon SQS charges are based on request volume and data transferred in and out of Amazon SQS. A consumer polling an SQS queue continuously results in empty receives. These empty receives are charged per Amazon SQS pricing even if messages aren’t sent or received from your SQS queue.
This may be obvious to folks in this thread, but wanted to drop this comment for greater context if some have no prior experience with SQS
f
Hey @Seth Geoghegan, yeah I set the
ReceiveMessageWaitTimeSeconds
based on the article to enable long polling. But i’m still seeing 15 empty receives per second 🤔
m
@Frank after some time did you see a drop?
s
Yeah, I recall poking at this value as well (per the AWS instructions) and don't recall seeing a drop in empty receives. It ends up only costing me $2 or $3/month, so I haven't bothered revisiting yet 🙂
m
For some context, we have a lambda trigger from sqs and I see a sum of 21k when the job runs. It's about 4800 items and they get chunked 25 per message.
I recreated the graph above and I see it's about 900/hr. Seems consistent.
f
Reporting back, it seems to have dropped to ~500/hr yesterday.. from a change that was made 4 days ago 🤔
Per minute it dropped from 15/min to 9/min… but if Receive message wait time is 20s, shouldn’t this drop top 3/min.. I’m afraid I don’t totally understand its polling mechanism