I was referring to this part from the doc (https://developers.cloudflare.com/workers/learning/using-durable-objects/#object-location): > Currently, Durable Objects do not migrate between locations after initial creation. Cloudflare will be exploring automatic migration compatibility in the future. But anyway I'll not bet on it for my solution 🙂

Just wanted to make sure you got understood it properly 🙂

Thanks 🙂 and thank you all for the suggestions!

Maybe im missing something lol, isnt it that 4 or more.. of the same workers could spawn an pick up 5 - " Cloudflare can instantiate multiple instances of that Worker within the same data center to handle the load."

the max concurrency is the max, it won't be more than that so if you say 5 it won't ever be more than 5 (the current default/max is 10)

This may be a silly question, but how long do Durable Objects last? Do they persist so long as a worker exists with a binding to it?

as long as they have storage and you dont delete them

I had the same issue. I wanted to migrate from our old system to Cloudflare and propagate changes from the old to the new, but only if the DO had been created. If it hadn't been created, I didn't want the old server creating the DOs around it. My final solution was to migrate the data ahead of time using location hints. All my users were in Intercom so I had latitude/longitude for them (roughly based on their IP, I'm sure that's how intercom pulled it). I went to Google Maps and put a pin in a "good location" to my untrained mind for each of Cloudflare's Location Hints so I could determine where a user was "closest" after comparing with each location hint. E.g. for Eastern North America I dropped a pin near Washington DC and for Eastern a pin near San Franciso. I also narrowed it down first by continent so I someone in Europe or Africa wasn't "closer" to a data center in the other continent. I figured they'd at least be on the same continent which better than all in Iowa USA where my old servers were. New customers will get the benefit of a server very close to them, but the rest will at least improve the location proximity unless they happen to be in central US, then it may or may not be. 🙂

Here is a gist with my solution: https://gist.github.com/jacwright/da95b48651c42f52263ef761e5df06c5

Hey folks! I'm designing an application that I think will use cloudflare for data storage/querying. I’ll use workers for the API endpoints, and my data changes infrequently, is extremely read dominant, and needs to scale to many millions of queries. There is little to no “search” or non-key-based querying, however there are quite a few one-to-many relationships, some of which will need to be “concurrently” editable. My initial thought is that I should use KV for data storage, using

type:guid

as keys and storing the data as blobs (I don’t need to query object values). If I do this, I’ll need some one-to-many indexes that wouldn’t be appropriate to store this way, so I’m wondering if I should put these in DO. My questions: Is this design incorrectly using KV or otherwise a bad idea? Are indexes an appropriate use case for DO? Outside of latency, is there a reason I should prefer density in a DO (e.g. putting multiple indexes in one DO) vs making a new DO per (object:relation) index? I assume using D1 just for indexes would be inappropriate, but if not, where would it compete favorably (e.g. cost, ability to query multiple index “shards” from a single worker)?

hard to say without knowing more about how those one-to-many relationships can change, but that seems like a reasonable use of KV

choosing DO granularity isn't just about latency, but also throughput - how many writes are you doing, are stale reads from KV or cache ok, etc

The one-to-many relationships are just associations. Imagine "editors" relationship between a post and user IDs. The indexes would always have keys of

guid:type

and values of lists of guids. They would change based on user-initiated actions, so would still be extremely read-dominant.

In general all the KV data is either append only or ok to just use last write wins even with stale data

Thanks for the spot check! I'm still trying to decide if I should do one DO per index or many indexes per DO - setting aside contention (I don't think this will be an issue, the data is pretty easy to shard), I think the main difference in my case would be that if I were querying for multiple indexes, I'd need to query multiple DOs which would be marginally more expensive due to the additional requests. So if I were to colocate commonly queried indexes together I could save some bucks down the line. There's no special rules around workers querying other DOs (e.g. if I have a worker which aggregates calls from DOs), correct?

I'm using a strategy in KV where I model 1:n relationships with n KV entries. I put all of the data in metadata rather than value so I can retrieve all of the entries in one list() request using the prefix option. Say you have Invoice with id "i1" and it has two Rows with ids "r1" and "r2". I would have two KV entries with the following keys "invoice/i1/row/r1" and "invoice/i1/row/r2". Then I can specify a prefix of "invoice/i1" in a list() operation to get back both invoice rows in one shot. If the info for the rows will fit in 1KB, then you never need to get() each individual row.

cannot access storage because object has moved to a different machine

this is happening quite a bit in our DO that we use as a queue, causing issues where it does a lot of work (that costs money in API costs) but we cannot save the result because the DO moved

is there a guaranteed timeframe where this won't happen? due to rate limiting we have to do a lot of retrying so it can cause the alarm handler to run for 2+ minutes

the shortest timeframe I've seen is just 32 seconds where this occurred after the alarm handler started

I would assume the normal 30 seconds?

Since that is also the runtime guarantee

alright, yeah it is happening less than 0.1% of the time for us (17 out of 17.3k in the last day) so will just handle it a bit better (not adding usage to the client) and let the other retrying layers take care of it

Invalid Durable Object ID. The ID does not match this Durable Object class.

I just bound a DO class to another worker, but now I get this error if I use the DO id created in another worker. (Worker A creates a DO, worker B tries to send requests to that DO) I thought this would work?

I can see in the dashboard that it's not the same binding. It created a new DO class, but with the same JS class. Maybe the

migration

in

wrangler.toml

should be different. Any doc on that?

So I needed to use

script-name

in the config

if you're trying to bind a DO that's published by another worker, you don't do migrations/export the DO code in the other worker

yup, use script-name and class in the binding

it's fine to query multiple DOs from a worker, but keep in mind DO throughput limits if you're doing read-heavy workloads. You may be better off sending all reads to KV, and doing writes through DO to KV

that is a useful strategy, but besides the 1kb size, also consider whether you're querying the same list repeatedly; if not it's never going to be in cache

Oh that is interesting, so I’d write to a DO and it would replicate to KV, and then I’d read from KV when stale reads are OK and DO when they aren’t?