After suffering last week from Vertigo (my tip, ju...
# general
s
After suffering last week from Vertigo (my tip, just don't !), it's time for the weekly poll 🙂 As a SaaS provider, are you running hardware per tenant, or perhaps they share resources with others? Please do elaborate in the comments how, why, etc. And yes, I know we all share virtual machines over physical hardware in the cloud, but AWS for example the load is indeed dedicated to the tenant for the time. Also interesting - is this permanently assigned, is it dynamic (how you do it)....
d
Dedicated infra is an option on Enterprise contracts, usually for security or performance customisation reasons; everyone else is shared infra.
m
@Daniel Chaffelson maybe dedicated, is a misnomer, I'm not talking about a business commitment to provision dedicated machines to a customer, but the architecture you built or when your code runs - is it processing a specific tenant's data/workload, or is it just serving random calls who knows who called? For example, say you have service called calculate, you might have an API that allows a customer to do a math computation (a+b), on it. You might route every call from any client to a centralized LB that distributes the load among the multiple instances of this very popular service - which handle the calculations as they come in (this is shared...) or the LB might be smart and start an instance (assume this is 0 time difference) - and send all a specific client to that specific instance(s), and calculations of other clients in other instances (the instance can be stopped when work is ended). So, this is more for tracking how much a client costs/uses, and security than about saving money and billing (i.e. without effective rate-limit per tenant, a single customer might still abuse my dynamic allocation system, but I'd know he used it... In both cases you can track client actions and bill activity, but in the Per Tenant you can also track resource utilization as well, and increase security (for example if you have a client secret, it will never be shared in the same runtime with other clients...)
d
ah ok, I better understand the question now. The architecture of our platform is inherently multi-tenant, in two main structures - for data processing and publishing it effectively uses row-level security with enforced filtering on a per-tenant basis, but they're all in the same general shared processing platform which uses components across Python / Redis / Clickhouse and a few other bits like security and routing layers - for the UI and CLI they are routed by their access Tokens to some shared serving infra and this essentially mutates the processing layer rules for their tenant(s). I think by your definition this puts us firmly in the Shared camp. For the enterprise customers I was talking about, essentially the entire architecture is instantiated for an isolated customer into some cloud provider zone, but that's really what you're looking for.
❤️ 1
m
What I'm looking for is an architecture definition that also works, my head is now on a container/function based code, that a routing layer launches with tags for every job/request, so that it's running in a separate runtime with it's own memory. But hosted on a shared cluster that is running all the tenants on the same virtual machines. We are also mutating the configuration per tenant, but right now - mainly because it was out of mind, and it saves money, we do everything in the same virtual environment. Specifically we do allow clients to run their own hardware (on their site or other) - on which they can host their own keys and logic, so we can't even do a configuration change or. sign a transaction in their name without them approving and signing on it.
c
We are working on: • Dedicated (your own VPC) • Reserved (reserved capacity in a shared cluster with a shared database) • Standard (fully multi-tenant) So all three. Pricing is dedicated most expensive, billed per-core Reserved is cheapest on a per-request basis, but it's billed per-core (so large cost overall) Standard is most expensive per-request, but it's basically serverless so generally small invoices.
❤️ 2
For the "Reserved" architecture, I think it matches what @Moshe Eshel is looking for. There is essentially going to be one big K8s cluster with a loadbalancer (Istio) that routes requests to the appropriate LittleHorse server owned by that client. The Dedicated is "whole arch set up for one customer" and the Standard is "one LH Server handlign requests for every client"
❤️ 1
d
It's a "it depends" question, but a couple of points I would make: • A lot of the time, the technology doesn't let you do shared resources at the item level (pool) effectively. There's no way to enforce isolation in the tech stack, so you have to do it in the application which isn't always what you want. • I see a lot of builders going for the bridge/mixed approach. Sharing the underlying resource (compute or data cluster) but giving each tenant their own logical resource (schema/DB-per-tenant, container-per-tenant). Mainly because they want to avoid complexity of isolation, scaling, or billing, or just having an existing product which is single-tenant-threaded. • It's a application-service by application-service decision. You may have a micro-service component doing silo because of a compliance reason, and another in the same micro-service pooled. • Having multiple offerings (like Colt mentioned above) is generally where I see things ending up. • It's a good idea to think about whether you can realise the efficiencies of a single architectural approach to isolation and access. If you can come up with a good pooled architecture, you can treat your silo deployments with the same architecture but restrict the number of tenants to 1. This can help simplify a lot of surrounding processes.
👀 1
💯 1
The last point doesn't always work, if you're adding a bunch of overhead to silo deployments without reaping a lot of benefits, it's just not in your interests. But there are definitely examples where it works. Then migrating a tenant to a different partitioning model becomes simpler, you can have the same control plane processes for scaling, billing, etc.
m
@Dave Roberts I think you're nailing it on the head, the answer is always "it depends" (anything that isn't can probably be automated anyway), what I like about this community is that we go into the reasons... I think your first point is the major argument, for most workloads - it doesn't make sense to segregate. • SaaS needs to be up and available, doesn't make sense to have the front end idling for a client that isn't even using it. • A technology that can handle this is something like Lambda or super lightweight containers - I think we aren't yet at the point where we can build & maintain complex systems with these effectively... The performance would not be good enough*. (but lambda/FaaS) has the granularity that's needed I do think the mixed approach is what works, or as you say service by service - but I find that most don't even consider the point and just follow what they did last time (whatever that may be) *a good DevEx product really if you can build it, a development environment - IDE, toolchains that allows you to write an application and on compilation/build converts it into much smaller pieces that are deployed...
🤘 1
m
Dedicated - run on our vpc with shared some things Byoc - fully dedicated