We're designing a system where: 1. A scheduled call to a 3rd party API from within a lambda call retrieves many thousands of records 2. Each of these records requires a call to another 3rd party API We want to be able to apply a global throttling to the the API calls in 2. Our thinking was that we'd write the records to a FIFO SQS queue so that the message group ID can be used to ensure that the same record isn't processed by multiple concurrent lambdas. However this doesn't solve the problem of being able to throttle the calls made by the consumers of the FIFO queue to the 3rd party API. Does anyone have any ideas of how we can design that?

Can you implement this with step functions?

I'll take a look, I haven't used step functions yet

hey Ross! can you just make a specific Lambda function whose only job is to call the 3rd party API, and set that Lambda function’s max concurrency to a really low number? https://aws.amazon.com/about-aws/whats-new/2017/11/set-concurrency-limits-on-individual-aws-lambda-functions/

^ that's the kind of stuff step functions enables. It's really powerful, took me a while to power through the docs but worth it

yeah. you shouldn’t need step functions to limit concurrency though

They help handle other things that can happen, like failures in one of the invocations

ah yes, the retry feature is awesome

Fyi the map step is what you're looking for: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-stepfunctions-readme.html#map Basically can specify the first function to return an array, then the next step maps through the array invoking a function for each (can control concurrency here) and then can do something with those results. Almost as easy as

get() |> map(fn) |> reduce

but with infinite scale!

Nice info gents, lots to dig into. Thank you!

@thdxr ha, nice 🙂 I have one step function with branches & parallels, and another that uses a wait task token to wait until a previous Lambda invocation says to continue.. super cool feature.

damn nice

we’ll also probably use wait tasks for the user signup flow, once we allow student discounts.. so the process will wait until we’re verified their student ID, then we click a button to approve, which signals the step function to proceed

that's super cool

This does look very cool

@Ross Coundon are you wanting to throttle them by 3rd party rate limit? Just implemented something similar in our app. At first i was trying to implement something like this: https://www.jeremydaly.com/throttling-third-party-api-calls-with-aws-lambda/ from Jeremy Daly, but too much moving parts and would be enough if you're jus dealing with one or two 3rd parties, on my case we were calling more. We ended going with eventbridge api destinations. Which is perfect cos it handle 3rd parties auth, throttling by rate limit, retry 429, 500 and content filter api to call by eventbridge rules. There are limitations though still like big one is it call and forget and there's no way to capture response yet. I think this is on aws roadmap. Our workaround was just call our own api as a proxy which then call 3rd party so we can capture response. But still same benefits of throttle and retries.

Very interesting, there's not a defined rate limit for the target system, but we know it's very sensitive to load so need to be able to tune our call rate

does the target API not have its own throttle where it’ll respond with a 429? and then you could just use an exponential backoff to retry?

Ha, if only