Does livekit cloud support providing more than 8GB memory or 4vCPU per agent?

our Scale plan has larger sizes then this currently; we can also talk about larger sizing if needed thru enterprise. We'd prefer to let you just think about concurrency though and not have to worry about specific resource sizing. are you running into issues or looking for something specific?

Thanks Dan, I see No issues but we're currently evaluating livekit cloud versus ECS for deployment so trying to understand the pros and cons in case our agent were to get heavier in future One thing I noticed is that it seems 'lk agent status' is not real-time and I don't think it tells you what timestamp the data is gathered, so it's difficult to know how much a given load takes in terms of CPU usage and RAM (as it's unclear at what timestamp the status was gathered), and I don't think you can view the CPU/memory usage elsewhere?

right, we don't surface that stuff anywhere. we do have agent join times in the cloud dashboard that show p90/p50/etc values to give you an idea of how it's performing. there is also a singular avg load metric on the dash

I see right, I tried a 20 user spike load test and found p90 of 60 seconds although I'm not sure I have the tools to diagnose if that's because too many jobs were allocated to a given node or because we needed to wait for more nodes to scale out or if it's just that a fully occupied node has insufficient memory/CPU

load tests don't usually model organic traffic. we do need some time to scale out additional replicas of your agent

It's true it only models likely something of a worse case, but it's important for us we know what to expect in the worst case so we can design our UX around it, for example if a 60 seconds wait is possible we would want to inform the user to wait up more than that many seconds before considering the agent not joining as a fatal crash and notifying support. Not only that but the "worst case" is something we expect when we have events when many people start using at a specific time. I assume there's no way of estimating how long it might be until an agent is available to join under sharp spikes in load, but one thing that would help my understand is whether livekit cloud starts with a desired count of 1 node and scale up incrementally? As that would imply that stepwise sharp increases in traffic (e.g. we have planned several events with 100 people that start at specific hour, so users would log on within about 3 minutes of each other) might take a long time to scale up I think For such an event is there any way to prepare high capacity (set desired count higher) for a temporary period?

i'm curious of your use-case where an event starts revolving around an agent joining individual rooms? we are working on improving cold start times to alleviate this issue; not sure of a specific timeline though. a specific endpoint to spool up max capacity is tricky since it could be abused quite often.

For context we have events whereby each attendee starts a session (an 1on1 interview with an agent) at roughly the same time (within minutes of each other), all 100 attendees need to all complete it by the end of the 1 hour session - as its 1on1 interviews its individual rooms for each agent. Is it right that this kind of setup, whereby you have around 100 people starting within a minute or two of each other, is not a use case currently supported by livekit cloud then? What sort of increase in users per minute is handled by current livekit cloud scaling policies? It would help me to understand what is the expected behaviour in a scenario like this - would the symptom be just a long wait time after dispatch occurs to when the agent joins? Or is there e.g. errors that would be transmitted to client on joining the room in this case (like "no workers currently available" kind of thing). For example, if we know it will take around 5 minutes to scale up capacity, we could put a UI message explaining to come back in aprox. 5 minutes, but if we do not have such an estimate, then its hard for us to determine if its some kind of bug (needing re-dispatching), and thus has hard to know how we would best inform the user or create a kind of "production ready" experience

ah interesting, was just curious about the workflow here. def sounds like a reasonable ask. i'll DM you in a bit