I’m after some advice from you excellent lot. We have lots of different business customers who have bought our products and sales are growing. Our main/flagship product has lots of different features and many many options for configuration of those features based on business requirements. Additionally, some of those features are part of the core product and some need a separate license. As we’re primarily using environment variables for configuration, what we’re starting to struggle with as that, when we introduce a new feature, or an additional configuration option for an existing feature, we need to update the environment variables for each individual deployment in Seed and redeploy. Some of our configuration is not env var-based and is done via JSON stored in parameter store and I’m starting to think that this is a better approach as it’s easier to write tools to auto update these values and they’d be picked up the next time a lambda cold-started (or we implement a TTL for values). This is all backend related but we have a similar challenge regarding our Vue frontends, the config for which is currently stored in individual config git repos and applied at build time to the core product code. I’m therefore curious as to what approaches others have taken to managing these kinds of changes.

Wouldn’t a feature flags system help with this? Maybe a dynamodb table with all customers and the features they have enabled, then both the backend and the frontend could check against this table what one cand and cannot do. Ideally you’d have a UI and you can enable/disable each thing at any time

My advice is don’t roll your own feature flag system (just like you didn’t roll your own framework/delivery system). LaunchDarkly is excellent. If you don’t want to go outside the AWS ecosystem, AppConfig has a newish feature flag offering. I haven’t used it yet, so don’t know how it compares.

Hey Ross, very interesting problem to solve, there are so many different ways, all having their own pros and cons! One thing I would try that I'd be interested to hear what you think about is - a separate repository with a resource stack that deploys these to your systems manager parameter store. This way, you have everything in a common repository for all customers, the features are declared in your main project stack, then the config is updated via this configuration repo/stack/project. Makes adding new customers a git commit, and updating flags a dependency as you have to update the new flags for the build to run on this repository. Forcing you to update the flags whenever you add a new feature, protecting you from forgetting about it. Let us know what solution you go for in the end as it's a very relevant problem for any startup.

I'd agree with @Matt Morgan on this. We had a similar issue in my previous company and an off-the-shelf feature flag system (there's plenty out there) worked well. Also means that as you grow, each Account Manager with the right permissions can also enable or disable these features for each client without needing any dev resource.

Thank you everyone. Feature flagging is on my list of things to investigate but had really thought of it as turning stuff on and off, rather than a fully fledged per-customer, configuration manager. Is LaunchDarkly the market leader here? It’s one of those things that follows me around the web so Google’s obviously reading my mind, but that fact rightly or wrongly often makes me wary of a product. From an implementation perspective, I assume you need to embed some switches in the code to make it work?

What about AWS App Config with Lambda Extensions : https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-integration-lambda-extensions.html

@JP (junaway) - that looks interesting, keeping it within AWS is quite attractive. Starting to think more deeply about this, we need to use the same feature flags at build time, as there are components of the app that will only be deployed if a customer has purchased that particular component. Presumably, if the configuration was in AppConfig, we’d be able to retrieve that in the SST stack definition and have it behave accordingly?

I'd use a mix of SSM and AppConfig depending on the type of flag. You might want to lay out all the criteria that matter to you, your company, your services, your opt-in features, and your clients. And then assign them to AppConfig, SSM, and env vars (because why not) depending on how dynamic or time-critical their update is.

Why not indeed 😄 It looks like AppConfig supports reading SSM docs anyway so that simplifies things

Mmh yeah but you'd pay the cost of both services, right? I know, I know, this comment is premature optimization. But keep that in mind.

I haven’t looked at the price yet 🙈

I may be wrong about pricing. 😄 You let me know went you look into it haha

LaunchDarkly is the market leader for runtime flags, but again I haven't looked at AppConfig. For env-specific build configs I have used the venerable https://github.com/node-config/node-config.