Hey guys, Is there any recommendation on how to organize S3 buckets? for example, multiple buckets, one for each feature, or subfolders by service, .. etc? Following the micro-services model, we create a bucket for each micro-service (in our case, we have services running on k8s). But also, I wonder if CDK can define properties to “backup” those buckets?… Want to hear ideas from you guys 🙂

I ran into S3 api call throttling issue a couple of times. (It’s a soft limit that you can request to get lifted.) Since then, I’ve tried to go w/ 1 bucket per feature.

Nice.

also keep in mind any authorization requirements you'll have. You can use the bucket keys in your security policies e.g.

/uploads/<user_id>

Regarding bucket backup… I was looking into this, https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_backup-readme.html

pre-signed url's have the access permissions of the creator of the url

So I never provide access to files directly, it’s always api-driven, and then user can hit the file (actually we use CFront signed).

which, might work perfect for your use case

Yeah I have to think about it.

I've seen the exact same thing

Yes! totally.

I've seen a single

uploads

directory that contained uploads from multiple customers in a multi-tenant system. The bucket was not public.

And also, people forgets about this, they just create a bucket when the project starts… and it just a resource that is never checked again while the application works fine.

However, every object within the bucket was public

Yeah, we have that problem here too.

so, nobody could list the contents of the bucket, but anyone could get a file if they simply guessed the object name

Hahahaha.

Regarding files being public by accident - cloud front OAI is a nice way to prevent it