What sorts of best practices are you using to mana...
# generals
Seth Geoghegan
12/10/2021, 7:08 PMWhat sorts of best practices are you using to manage long-lived resources (e.g. RDS, VPC, ElasticSearch, etc) in ephemeral team development environments?
Seth Geoghegan
12/10/2021, 7:11 PMIf I am developing a REST API with API Gateway, Lambda and DynamoDB, it's fairly straightforward to spin up individual resources for each developer when they execute
sst startpackage.json"scripts": {
"start": "sst start --stage $(whoami)"
}
t
thdxr
12/10/2021, 7:12 PM^ you don't need to pass in whoami anymore fyi
thdxr
12/10/2021, 7:13 PMIt'll generate a unique stage on first run and ask you to confirm or change it
a
Adrián Mouly
12/10/2021, 7:13 PMI did that before until @thdxr added the helper.
Adrián Mouly
12/10/2021, 7:13 PM😂
s
Seth Geoghegan
12/10/2021, 7:15 PMahh, that's right!
Seth Geoghegan
12/10/2021, 7:15 PM@Adrián Mouly I saw your post a while back around how you decide to define a resource or not
Seth Geoghegan
12/10/2021, 7:16 PMusing SSM/ARN's, and I believe @Ross Coundon was using environment variables to a similar effect
a
Adrián Mouly
12/10/2021, 7:16 PMYeah.
s
Seth Geoghegan
12/10/2021, 7:16 PMI think some shared resources make a ton of sense (e.g. a VPC everyone uses for development)
a
Adrián Mouly
12/10/2021, 7:16 PMI’m sing SSM to store the ID of the VPC and the Secgroup.
Adrián Mouly
12/10/2021, 7:17 PMYeah, exactly. We create those only once.
s
Seth Geoghegan
12/10/2021, 7:17 PMbut it's not so clear to me that it's a good idea to share something like ElasticSearch or RDS
Seth Geoghegan
12/10/2021, 7:17 PMIn the case of RDS, I used a local DB 🙁
a
Adrián Mouly
12/10/2021, 7:18 PMFor RDS I was tyring to use VPN, but I’m still in the process.
Adrián Mouly
12/10/2021, 7:18 PMES also doing the same as VPC/SG.
Adrián Mouly
12/10/2021, 7:18 PMWe create it only once, and then reference it from SSM.
o
Omi Chowdhury
12/10/2021, 7:20 PMYah, for me it comes down to cost and complexity of setup through CDK. If either of those are high, then sharing via SSM makes sense.
We share a cognito pool in dev, and some external SaaS APIs
s
Seth Geoghegan
12/10/2021, 7:22 PMI guess I'm wondering about instances where usage of the shared resource might overlap, like two developers creating/updated ES indexes in the same ES domain. For example, I might create a list of movies to search while a team member updates/deletes the same list of movies.
o
Omi Chowdhury
12/10/2021, 7:23 PMYeah ideally shared resources almost never change their configuration - if they do then its definitely an issue
s
Seth Geoghegan
12/10/2021, 7:33 PM@Adrián Mouly How do members of your team use the shared ES resource in a way that doesn't interfere with one another's work?
a
Adrián Mouly
12/10/2021, 7:40 PMGood question…. we actually didn’t do too much with ES at the moment, we have a single guy who is the “expert” and I think he did some configurations connecting to the ES service … not from code 😞
But well, the ES instance is created from code, nothing else.
d
Damjan
12/10/2021, 7:40 PMIf you have more than 3 people working on anything, you’ll spend way more money debugging random issues vs just paying for separate environments
Damjan
12/10/2021, 7:41 PMSo unless it’s impossible (with certain SAAS), have different tenants/services for everyone
s
Seth Geoghegan
12/10/2021, 7:42 PMI'm introducing SST to an organization with 40+ engineers and I'm trying to figure out the best way to advocate for development in the cloud (they currently lean heavy on docker and local emulation) Spinning up 40+ ES domains (or RDS instances, etc) might be a tough sell
Seth Geoghegan
12/10/2021, 7:43 PMIt's an easy sell with services that scale to zero (DDB, API Gateway, Lambda, etc), but seems to require a different approach with services that do not operate that way
d
Damjan
12/10/2021, 7:44 PMI think it is … but only about 20 devs in my team.
But we are also trying to move away from stuff that requires a VPC.
Damjan
12/10/2021, 7:44 PMYou can always have indexes/databases per dev, it’s just a drag and more complexity.
s
Seth Geoghegan
12/10/2021, 7:46 PMHmm, interesting. I hadn't thought of specific indexes per dev (I'm fairly new to ElasticSearch)
Seth Geoghegan
12/10/2021, 7:46 PMBut I agree with you, I prefer clean and simple
Seth Geoghegan
12/10/2021, 7:47 PMjust trying to explore the pros/cons and various options. I know someone will say "Just use Localstack", then I have way more problems 🙂
d
Damjan
12/10/2021, 7:50 PMYeah, but rolling out something new to 40+ people, the last thing leadership should worry about is minuscule cost per month per develop, when devs will burn through thousands via their time trying to work towards cost optimisation 🙂
Damjan
12/10/2021, 7:51 PMIt’s always the same though, let’s save no matter how much it costs 😉 So smart to think about it and working through options.
s
Seth Geoghegan
12/10/2021, 7:53 PMYeah, tricky to navigate situations like this. People can react negatively to change, especially if it requires a paradigm shift in thinking about how we develop software. Doubly so if it saves $ (or sometimes, even just a few ¢)