Yeah, if you're OK with losing a little, then you could set a timeout to persist state every N unit time or something

It would be cool though if there were a way to fire a handler whenever the runtime knows that it will soon be evicted, even if that won’t cover every eviction case

is there ever an eviction while a request is active? If not, is there a way to extend that behavior a bit? like with

ctx.waitUntil

?

I think our fear is that people will use it assuming it's reliable, but I do agree it'd be a nice to have. Another issue is how long to give someone to do extra work. Also if you mean just eviction because the object is idle, you could implement your own with a 30 second timer after the latest request, and debounce it anytime a new request comes in.

That's exactly what I was thinking

Depends how you define eviction. Since hardware crashes you can't always do stuff when a DO instance it being killed. The only other time an instance becomes "broken" is if there is something like an error accessing storage, or if you do a code update and try to use storage.

I guess I mean eviction not from failure like hardware crashes, but standard eviction behavior like when worker updates are deployed

The use case I was playing with was to keep in memory state with a 20-30 second timer in ctx.waitUntil ... and write to KV if another request doesn't come in ... for very large & changing datasets the pricing of DOs is really expensive - so for some of these use cases we're writing to KV from DOs ...

Maybe the function it calls can just be like unsafeEviction if you guys do add it

Yeah I think the timer thing I mentioned above would cover you pretty well except for code updates. We went back and forth on updates when we were designing the system

It could be cool to have a "let the old instance finish up work before starting the new instance" but that does imply requests would need to block until that work finished

If I know that all of my requests are hitting essentially 1 colo, it would be faster to have a durable object + some cache store things in that colo over KV, right?

Probably, as long as that colo supports DOs, that the values are small, and that you wouldn’t overload the DO with unique requests.

All I'd use the DO for would be a cache essentially - the rest of the processing could happen in the normal worker. But yeah I'm storing like <1kb values

(as for "why not the cache api?" - the less common writes can come from anywhere, so I need it to update in this colo asap)

If all invocations were in a single colo, then would you even need a DO?

i guess if you have conflicting writes ...

A DO in the same colo would probably respond faster than a KV origin in a completely different colo

Writes (much less common) can come from anywhere, reads will come from just one place

I estimate read-write to be something like 100:1

I assume you're OK with the cache value being evicted

It's even possible (though unlikely) your DO is in Colo 1, has set a Cache value of A. Then that colo has issues and we move the DO to colo 2, where it sets a Cache value of B. Then the DO moves back to Colo 1 where it reads a Cache value of A.

And your normal Worker requests can be routed to different places for many different reasons, so I don't know if you can really depend on always hitting the same colo as your DO just because you observe that to be the case right now

I'm basing this on the fact that about 99% of requests go to IAD at the moment, so putting my cache in a DO in IAD would be beneficial for performance over just using KV.

And yeah, eviction is fine, I don't expect these values to change often, but when they do, I'd like the cache up to date asap (which is why the cache API is not ideal - the update could come from anywhere, as opposed to the read, just IAD)

We have a number of colos in IAD that will have their own colo caches

It seems like your normal Worker could try cache, fallback to DO, then put to Cache if it wasn't there? Depends on the details, I guess

It's a bit of a tricky one, isn't it 😅

A very specific need, will have to think more about it

Oh interesting ... how many colos have independent caches? Would that tend to be the same colos that have enough redundancies to also support DOs? Or is that an outlier given how core IAD is?