https://discord.cloudflare.com logo
Join Discord
Powered by
# durable-objects
  • b
    You could synchronously manipulate state in memory and then persist changes to storage, as seen in our counter example: https://developers.cloudflare.com/workers/learning/using-durable-objects#example---counter
  • v
    this does not work when the data is too large to hold in local state
  • v
    for our usecase our USER DO holds the users "jobs" which are 4kb in size each describing a bunch of metadata
  • v
    we need to do a Read/Update/Write
  • v
    if a user has more than 100,000 jobs, that will be more than the memory limit on the DO can handle
  • b
    We'll likely improve the storage API, but in the meantime you could store a shared promise when you're updating a job, and requests would check and wait on the promise before they themselves do an update, effectively serializing them. You could them per job-id, too.
  • v
    but its a good point.. i think wel consider this actually 128MB is plenty of wiggle room, and we could offload the cold archived jobs
  • b
    e.g. Check 
    ongoingUpdates[jobId]
    for a promise, if there is one, wait for it to settle, then put your own promise in there.
  • v
    yea per-job-id would be perfect. im having a little trouble to understand how to do this
  • v
    i see the idea thought, but unsure how the code will look
  • v
    yea here we go
  • v
    Copy code
    {
   error: "crash",
   stack_trace: "Error: Transaction failed due to conflict while processing a put() operation."
 }
  • v
    this transaction fix wont seal the deal 😦
  • v
  • v
    basically a bunch of concurrent jobs going on (other DOs) pushing updates to 1 DO the USER
  • v
    would very much appreciate a pseudocode snippet on this promise idea, not sure how the code should look. il move to implement like the counter example, so we just need to update/write no need to read. and limit the users jobs to like 10k, should be more than enough.
  • b
    Copy code
    while (true) {
    let ongoingJob = this.ongoingJobs[jobId];
    if (typeof ongoingJob === undefined) {
        let promise = asyncFunctionThatDoesProcessing().then(() => {
            // Success, remove ourselves from the ongoingJobs object.
            delete this.ongoingJobs[jobId];
        }, (err) => {
            // Error, remove ourselves from the ongoingJobs object.
            delete this.ongoingJobs[jobId];
            
            // Report the error somewhere, or throw if you want other requests to see it.
        });

        // Store the promise so other requests see it and await.
        ongoingJob[jobId] = promise;

        // Await our own work.
        await promise;

        // Break out of the loop, we're done.
        break;
    } else {
        // Wait for someone else's work to complete.
        await ongoingJob;

        // Loop back around.
    }
}
  • b
    That's untested pseudocode 🙂
  • b
    The basic idea is that inside of a Durable Object, different requests can await the same promise. So that should be serializing your requests per jobId. You need to be very careful when doing this that the promise is removed when it resolves or rejects.
  • b
    It's similar to the initialization promise in the Counter example
  • u
    Just curious is it possible to put a ttl on durable objects?
  • k
    Not yet. But if you don't store any data (i.e. using state.storage.put()), the object will delete itself when it shuts down. So if you only need it to last a minute or so, store data in memory only and you're good. If you need something that persists for more like an hour or a day, you'll need to wait for us to implement scheduling.
  • u
    Hmm okay yeah I am basically just making a small game that the data only lasts while players are actively playing. So if the object doesnt get deleted so long as there is an active socket connection that would be fine. I wasnt entirely sure if this was the case or not because of cpu limits it might restart?
  • k
    Well, it might restart randomly. Not often but occasionally.
  • k
    not just because of cpu/memory limits, but also if we're rolling out an update to the workers runtime (which we typically do a few times each week), or because load needed to be redistributed, or a network issue, etc.
  • u
    Hmm yeah that should be good enough I can just add in a small thing on the host to send all the game data at its current state if it restarts for whatever reason. That will make it a bit easier since I do not care whatsoever about having this data persisted 🙂
  • v
    @User that code above i just get, {"error":"internal error","stack_trace":"Error: internal error"}, and CPU time exceeded
  • v
    and im unable to understand whats going on enough to modify it
  • v
    {"error":"Durable Object exceeded its CPU time limit and was reset.","stack_trace":"Error: Durable Object exceeded its CPU time limit and was reset."}
  • u
    @User I assume its possible but with itty router and durable objects is it possible to do websocket? Adding an example with WS to https://github.com/kwhitley/itty-durable-example would be really cool if it is possible and you have time 🙂
1...727374...567Latest