https://serverless-stack.com/ logo
#help
Title
# help
s

Seth Geoghegan

01/04/2022, 5:55 PM
How are folks managing resources within an application that have distinctly different lifecycles? I have an API that uses an RDS instance. The RDS instance has stages
dev
stage
and
prod
, all are meant to be long-lived. The API has long lived stages
stage
and
prod
with an ephemeral local environment named after the person doing the development (e.g.
sst start --stage {whoami}
) I want the ephemeral API environment to use the
dev
RDS instance. The same problem applies to any long-lived resources (RDS, ElasticSearch, VPN's, EventBridge, Route53, etc). I tried and tried to get this working within the same SST app by conditionally creating resources depending on
app.stage
, but it felt forced and error proned. I read The CDK Book looking for clues of useful patterns, but I didn't find what I was looking for. For now, I've created two SST apps; one for the RDS Instance and the other for the API. This feels much more natural, but I'm wondering if I'm "Doing It Right™️"?
s

Sam Hulick

01/04/2022, 6:16 PM
I’m probably not going to be able to answer your question sufficiently, but.. if a dev is running
sst start
, you can use
process.env.IS_LOCAL
to determine whether to use
dev
RDS or not. (since you can’t check
app.stage === 'dev'
obviously. as for conditionally creating resources based on stage, there’s nothing wrong with that. I have a few spots in my code where I do that.
most of the stack is the same code for dev vs prod, except for things like domain names, or other specialized stuff I don’t want to have in dev, but do want in prod
s

Seth Geoghegan

01/04/2022, 7:51 PM
Thanks for the insight! It's not only conditional creation, but removal as well. If the database does not exist, I want to create it. If it already exists, I do not want to remove it when the stack is destroyed. Making sure all this logic is correctly implemented seems tricky enough that I feel like I'm not using it as intended. All of that complexity around checking for
process.env.IS_LOCAL
and
app.stage
goes away if they resources live in separate apps with their own independent lifecycles. While that does introduce complexity around deployments (e.g. the DB app needs to be deployed first), that seems more manageable.
m

Marcin M

01/04/2022, 8:51 PM
I am not sure if it solves your problem, but we do distinguish between application infrastructure (which is, let say fixed) and application itself (the things that the devs can change/add/remove like api, its routing, lambdas, step functions, etc)
(the third layer is account setup)
s

Seth Geoghegan

01/04/2022, 8:53 PM
Yes, that concept is what I'm trying to implement via CDK/STT. How do you manage the application infrastructure vs the application itself?
s

Sam Hulick

01/05/2022, 2:34 AM
@Seth Geoghegan separate folders, and yarn workspaces. so for instance, I have
infra/
where all my infrastructure code lives, and
src/
for the application code. and
src/package.json
contains the packages that are intended only for use with the application itself. root-level
package.json
contains:
Copy code
"workspaces": [
    "src"
  ],
s

Seth Geoghegan

01/05/2022, 2:36 AM
Funny timing, I was just reading some old threads of your use of yarn workspaces! I only have a basic understanding of them, but it sounds like I need to read more. What does your
/infra
folder contain? An SST app?
s

Sam Hulick

01/05/2022, 2:46 AM
yep!
m

Marcin M

01/05/2022, 8:51 AM
@Seth Geoghegan by projects separation
s

Seth Geoghegan

01/05/2022, 12:47 PM
@Marcin M can you elaborate on what you mean by project separation? Do you mean you have multiple SST apps? Are they all in the same repo, or are they independent from one another?
SST enforces a stage per app, which is making these approaches difficult (impossible?). The resources are related, but are deployed to different stages at different times. I don't see how to do this cleanly within the same SST app.
Visually, this is what I'm trying to do. I do not think can be done within the same SST application due to how each sst.App has a fixed stage that cannot be modified. I don't think it's possible to deploy one resource to a
dev
stage, while deploying other resources to a
seth
stage. Is my understanding correct?
s

Sam Hulick

01/05/2022, 6:22 PM
@Seth Geoghegan looking at your diagram. Sam/Seth/Frank shouldn’t be sharing a single DB, IMO. dev RDS would be moved into their respective AWS accounts, which is how our setup is. that way, devs can really go off the rails & experiment freely without affecting other devs.
s

Seth Geoghegan

01/05/2022, 6:40 PM
My intention is to share a single RDS instance with different databases per developer. That way, each developer can still have their own encapsulated database and we don't need to pay for multiple RDS instances (both in time to provision and $$$ to operate). I'm trying to support an engineering organization with dozens of developers and we do not have individual developer accounts 😕
s

Sam Hulick

01/05/2022, 6:40 PM
@Seth Geoghegan ahh I see. so all the devs are in a single AWS account?
s

Seth Geoghegan

01/05/2022, 6:41 PM
Yeah, unfortunately 😢
s

Sam Hulick

01/05/2022, 6:41 PM
we have it set up so there’s an AWS organization that contains an AWS account for every dev. and we’re using Aurora Serverless, so the dev DBs spin down after an hour of being idle (to save $)
s

Seth Geoghegan

01/05/2022, 6:41 PM
ahh, yeah, the Aurora Serverless option would definitely help with this problem
I'm fighting that battle too 🙂
s

Sam Hulick

01/05/2022, 6:42 PM
good luck! 😄 sorry I can’t be of more help
s

Seth Geoghegan

01/05/2022, 6:43 PM
You've definitely helped, thanks for walking through the thought process with me
s

Sam Hulick

01/05/2022, 6:43 PM
you’re welcome!
m

Marcin M

01/07/2022, 9:04 AM
we have two repos, both containing SST apps. I am not sure if it going to stay like this in the future. We are convinced to split the app from its infrastructure. As we are learning that stuff right now, we think that it should be split, but maybe the infra does not have to be described as STT App ( for example, could be terraform scripts)
s

Seth Geoghegan

01/07/2022, 2:06 PM
Thanks for chiming in @Marcin M, I had a similar thought this morning. My organization makes heavy use of Terraform, so an
infra
directory in my SST app that contains terraform scripts to generate the RDS Instance could certainly work. While I certainly can provision an RDS database using SST/CDK, I'm not really using any of the SST constructs or the live Lambda development environment. Using an entire SST app to provision a single RDS database doesn't seem like the sweet spot for SST. I think the lesson I'm learning is that there is a difference between long-lived infrastructure and infrastructure I create/destroy at will. Forcing these two types of infrastructure to be within the same SST app (and stage) feels counterintuitive and can be a real pain to manage!
2 Views