Hey guys.
I have a NestJS app running on a k8s cl...
# ssta
Adrián Mouly
03/23/2022, 7:46 PMHey guys.
I have a NestJS app running on a k8s cluster, all fine.
But I want to hook these services to an SQS queue.
The way I found to do this is:
1. Doing “polling” of the queue all the time
2. Adding a lambda triggered-by-sqs which can call my Service.
3. What else? is there a way to hook an endpoint directly to SQS?
a
Adam Fanello
03/23/2022, 7:55 PMI haven't done this, but long polling looks like what you want. https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/sqs-examples-enable-long-polling.html
a
Adrián Mouly
03/23/2022, 8:14 PMI don’t want to do long polling.
Adrián Mouly
03/23/2022, 8:14 PMIf I can do a lambda trigger + webhook.
Adrián Mouly
03/23/2022, 8:14 PMRight?
a
Adam Fanello
03/23/2022, 8:47 PMSure. That sounds like something that will add cost, complexity, and latency. The k8s cluster is running and consuming resources. All the Lambda AWS Service does is long poll the SQS queue, and then trigger your lambda when something is returned, which will then trigger your code in k8s.
Maybe polling from within NestJS isn't possible?
a
Adrián Mouly
03/23/2022, 8:53 PMIt is posible.
Adrián Mouly
03/23/2022, 8:54 PMBut I want to avoid polling.
Adrián Mouly
03/23/2022, 8:54 PMSQS already has hooks/consumers for Lambda.
Adrián Mouly
03/23/2022, 8:54 PMSo the chain will be…. SQS -> Lambda -> k8s Service (nestjs)
a
Adam Fanello
03/23/2022, 8:55 PMI understand. I am pointing out though that Lambda Service (not your function) makes that happen by long polling. You may not be doing the polling, but it is happening.
a
Adrián Mouly
03/23/2022, 9:04 PMI see.
Adrián Mouly
03/23/2022, 9:04 PMI don’t think is like that.
Adrián Mouly
03/23/2022, 9:04 PMI think is SQS calling Lambda, the opposite?
Adrián Mouly
03/23/2022, 9:04 PMBut not sure, to be honest.
Adrián Mouly
03/23/2022, 9:04 PMIn your concept, “somebody” is doing the polling, in low level.
Adrián Mouly
03/23/2022, 9:05 PMI thought SQS had a list of “listeners” and will call each of them, in this case, a lambda-listener.
Adrián Mouly
03/23/2022, 9:05 PMLike… 1 message comes in -> deliver to all the listeners.
a
Adam Fanello
03/23/2022, 9:05 PMSome Lambda integrations get called, and some poll. EventBridge calls lambda. Streaming data sources, like SQS, Kinesis, S3, dynamoDB, (and more!) work by way of AWS doing the polling for you.
Adam Fanello
03/23/2022, 9:06 PMThis is just implementation detail, but you sound hung up on not wanting long polling. I don't know the reason for that, but just pointing out that many Lambda event sources do work that way.
Adam Fanello
03/23/2022, 9:08 PMA reason in favor of putting Lambda in between SQS and your NestJS containers is to decouple your system. Maybe you want to be able to hit the API in NestJS from other places (like the client?), and so let that stand alone and the (other source) => SQS => Lambda is a separate thing that your NestJS app doesn't need to know even exists.
Adam Fanello
03/23/2022, 9:08 PMThat does bring up a possible security aspect. If you go the NestJS HTTP API route, is security of that endpoint an issue?
Adam Fanello
03/23/2022, 9:11 PMt
Tim Birkett
03/24/2022, 4:27 PMIs SNS helpful here? https://aws.amazon.com/sns/
Tim Birkett
03/24/2022, 4:29 PMYou can use SNS and SQS together to create a range of messaging patterns.
f
Frank
03/25/2022, 2:13 AM
Second @Adam Fanello here. Lambda polls from the Queue behind the scene. Polling doesn’t necessarily mean delays if that’s ur concern. U can poll in a way that meets ur latency requirement ie. long poll vs short polling. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html
11 Views