https://serverless-stack.com/ logo
Join Slack
Powered by
in the Queue tutorial, maybe you should mention to...
# help
a
in the Queue tutorial, maybe you should mention to set the following, because otherwise there will be costs generated by the large number of empty receives:
Copy code
sqsQueue: {
   receiveMessageWaitTime: Duration.seconds(20),
},
hmmm, actually, this doesn't work, let me check the correct setup [Edit] Fixed the code above
t
what do you mean empty receives?
a
I just got a free tier alarm that my million requests are almost used up, and I checked the sqs monitor, and all that's in there is high values for NumberOfEmptyReceives
i read up a bit on it and it seems the consumer is constantly hitting the queue and if there are no messages in there, an "empty receive" is generated
t
I don't think that's exactly right
a
i get 20 such empty receives per second
t
since this is a managed integration, your lambda should only be invoked when there's a non-empty receive
there's no mechanism to charge you outside of your lambda being invoked
if you look at your lambda invocations you shouldn't see 20 per second
a
yeah, it might be a problem of my setup, I guess, although I used it almost exactly like in your queue example
t
The free tier alarm - this is the limit for lambda invocations? I suspect something else is triggering
a
no, it's the sqs requests
t
ah that makes more sense
a
honestly, it's a bit weird, because even with the max long polling config (20 seconds) it's going to create over 200k empty receives a month
t
can you show me a screenshot of the graph
a
so really your free sqs tier is below 800k requests, and not the million they claim
t
want to compare with my own queues
d
Can confirm all this is correct @thdxr. At a 20 second wait time an SQS/Lambda pair generate at least 131,400 requests / month if no messages ever show up (Long Poll). With short poll, its about 20x that.
a
btw, this is the amazon article that i found when looking for the problem: https://aws.amazon.com/premiumsupport/knowledge-center/sqs-high-charges/
t
yeah my chart is half of yours
but that's likely because it's just a test one
a
@thdxr what config did you use?
t
Just the defaults
a
and you never got a free tier alarm?
because even if you had half of my requests you would run out in under a month
t
I don't really have much in production
a
ah, ok
t
no I don't think I got an alarm
a
in the billing page, how many sqs requests do you have for the current month? mine is already at 850k
t
maybe I don't have billing alerts set lol
d
do you have more than one lambda hooked to the queue @Adrian Schweizer?
a
nah, that would keep you under the 85% alarm for the month
@Derek Kershner yeah, there were 2 at some point
d
that would double the amount of requests as well.
a
still thdxr's amount is less than half of mine
I did have some legit production requests, too, though, so that might account for the difference
t
there's your fix, never actually ship anything to production
a
haha
d
i know several companies that take this tact.
a
nah, it's no big thing, even if I had passed it this month it wouldn't have cost much
but still maybe a good idea to add it to the Queue tutorial, or what do you think?
or maybe a link to that amazon page?
t
it might make sense for us to set that as the default - is there any downside to increasing that to the max of 20?
d
I dont know of one. At high volumes, short or long polling would mostly be limited by batchSize, not poll length.
t
yeah
weird that it's not the default to use long polling
d
the cost of SQS is the same, and AWS itself eats the wait time.
t
also weird that under the hood they don't re-architect things to be more "push" driven
d
well, SQS is the “pull”, SNS covers “push”
t
From our point of view yes. But if they're providing a fully managed "polling service" through the lambda integration it's weird they don't implement that as push based under the hood
I know there are some complexities but AWS knows when the queue has a message, they know which lambda integrations need to be triggered
could just trigger the poll when the messages show up in the queue
d
perhaps true, but this way they get to push the costs on to us instead 😉
also, I think they tend to think in large volume terms, when none of this would matter.
t
right
a
I guess the only downside is that the consumer lambda can wait up to the specified time before returning, which I guess is unproblematic in most cases
t
I don't think you'd see any change in your lambda
a
well, it certainly won't be a problem in my case, because I'm just using it to send reminder emails
d
If you mean that the lambda may not trigger for up to 20 seconds, this is correct, but Dax is correct that the actual lambda execution time wont change.
a
hmmm, unfortunately, it seems the config didn't help much, the graph didn't go down the last few minutes
d
Check the lambda trigger config, make sure the 20 is present.
a
yeah, it's there
I guess the empty receives I'm still getting are from other projects
or rather other deployments
hmmm, no, it's making a separate queue for all deployments obviously...so wtf is going on?
that is over 650k requests per month
that's a bit shit
d
you know what? that supports my experience, I just figured it was volume pushing up concurrent lambdas, but it never dropped below 5. Didn’t know why.
a
I mean, it's still very cheap of course, but only 2 lambdas makes it non-free
i mean only 2 consumer lambdas
d
I think a FIFO queue might only spin up one poller…
a
let me try that
d
its $0.10 more, but if so, it would actually be less at very low volumes
a
great, now it can't update the queue, because it's already in use by my api (orly?)
c
I also receive the SQS free tier alert like 3 days before the month is done
a
i think in my case, instead of an SQS triggered lambda, I could actually do a cron triggered lambda that manually reads from the queue
d
just make sure you are OK with messages sitting for awhile, and set an alarm to detect accumulation, but nothing wrong with that.
also make sure NOT to long poll in that case
a
could still long poll, since it's only 20 seconds
d
better not to, so that lambda execution time is much lower on zero messages.
a
ah, yeah, that's true, thanks
l
I've seen people circumvent the whole empty polling issue by not having any consumers and listening to cloudwatch
Cloudwatch shows entries in queue - push trigger to lambda that manually pulls from the queue and deletes entries on success
t
lol
crazy
l
Kills the entire idea of easy queues but if you've got loads and are actually paying for them could lower your cost
I also stumbled upon free tier alerts when we went from 1 simple sqs to an honest set
Then set the defaults in my stacks to long polling
But yeah, AWS could actually lower our costs by default if they only used what is possible with cloudwatch instead of just pulling 😅
a
@Lukasz K that is a neat idea, thanks for mentioning it
I also thought about a CronJob that checks the queue, but that would of course still give you some empty receives (but way less, even if run every minute)
checking the logs instead is a really cool idea
Open in Slack
PreviousNext
Powered by