Curious how everyone is handling deprecated Lambda...
# aws
m
Curious how everyone is handling deprecated Lambda runtime upgrades at scale? We have a few hundred functions running across multiple AWS accounts and looking for some creative approaches
šŸ’” 2
m
I don’t have any first-hand experience to offer but I’m loving the scope of this question! In my experience, I’ve just updated one-off functions here and there….never thought about hundreds at a time. questions: 1. Are they all production environments or is there a mix of non-production?is CI/CD in place for most, many, all functions? 2. Are they all the same runtime or is there a mix? 3. What does function execution look like for services that call and/or depend on the functions? 4. Based on 3, is lift-and-shift an option (to a new lambda shell with a supported runtime) and for any functions accessed over HTTP, update the endpoints? 5. What’s the timeline for the change to be in place? I’m eager to follow along!
m
Right? I've had a similar approach in the past, but just inherited a fragmented environment with a mix of functions that are managed by a bunch of different teams. Each team manages the Lambda resources in different ways from CF to TF to clickops (it's a mess). They're a mix of runtimes from old Python versions through Java 1. No hard timelines yet, but hopefully before anything breaks lol!
šŸ¤¦šŸ¾ā€ā™‚ļø 1
My first step in the identification piece is using Trusted Advisor to identify all of the deprecated runtimes. Was thinking of creating a Cloud Custodian to tag these resources for easy filtering in the future
m
Yeah nice start. Gotta know what the scope is. After that I would work from oldest to newer and maybe collected by runtime. If you have SMEs you can reach out to them for input on risk due to changes. That's my biggest fear with these kind of updates.
this 1
Or hey, run each function through AI and ask what the dangers are with a change in runtime from X to Y.
m
Was able to make a cool Cloud Custodian policy for it too!
Copy code
policies:
  # <https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html>
  - name: lambda-deprecated-runtime
    description: |
      Tags Lambda functions that are using a deprecated runtime as noncompliant.
    resource: lambda
    filters:
      - type: value
        key: Runtime
        op: in
        value:
          - nodejs20.x
          - ruby3.2
          - python3.9
          - nodejs18.x
          - dotnet6
          - python3.8
          - nodejs16.x
          - java8
          - go1.x
          - provided
          - ruby2.7
          - nodejs14.x
          - python3.7
          - dotnetcore3.1
          - nodejs12.x
          - python3.6
          - dotnetcore2.1
          - nodejs10.x
          - ruby2.5
          - python2.7
          - nodejs8.10
          - nodejs4.3
          - nodejs4.3-edge
          - nodejs6.10
          - dotnetcore1.0
          - dotnetcore2.0
          - nodejs
    actions:
      - type: tag
        key: CSE.Compliance
        value: NonCompliant
      - type: tag
        key: CSE.Check
        value: LambdaDeprecatedRuntime
šŸ”„ 1
m
Nice! I wonder if there’s a way to create custom metrics from the output so you can create a CloudWatch dashboard that is a self-updating burn-down list: Drive the ā€œCount of functions with deprecated runtimesā€ down to zero with breakouts of counts for each runtime. I report stuff like this to leadership regularly so having a dashboard that I could just pull from to create a slide would be my first priority šŸ˜…
e
LMK if we can add an
aws/lambda
component type! šŸ˜‰
Wouldn't the first step be to consolidate the deployment patterns?
Then centralize configuration of runtime into a versioned config
Then treat this as an exercise in drift
> Curious how everyone is handling deprecated Lambda runtime upgrades at scale? ...said another way, this issue is the symptom of all the other problems you pointed out.... > a fragmented environment with a mix of functions that are managed by a bunch of different teams. Each team manages the Lambda resources in different ways from CF to TF to clickops (it's a mess).
Thus, I would challenge the team if it's reasonable "at scale", that each team manages the Lambda resources in different ways from CF to TF to clickops
Or if instead, a more deliberate deployment strategy is merited.
m
An Atmos Custodian custom command would be pretty slick šŸ‘Œ 100%, the historical context is that a consultant came in and deployed a bunch of Lambas without leaving a trace of IaC and I'm tracking down how the other teams are managing it. I'm in a unique situation where I've inherited the security of an organization that didn't really practice it, so I'm trying to balance making it easy for teams to practice security and do self service without taking over all their code. I've been able to diagnose where they are, now it's a matter of getting the policy to tell teams to upgrade and enforcing it
e
That is really good context
And a big job!
Yes, I think it prudent not to rock the boat too early, and make the smallest changes as you are to get things to a stable state (and then rock the boat!)