I was forgetting about "DurableObjectStub" which extends Fetcher

that was my missing piece. Thanks so much for bearing with me 🙂

Yeah - you'll see

Fetcher

used around the place for Service Bindings, mTLS certificates & Workers for Platforms They all expose a

fetch

method, different to the global one in some way. In DO's case, it sends the request to your Durable Object. In Service Bindings (and WfP) case, it sends it to the bound Worker. In mTLS certificates case, it'll present a client certificate with the request.

The URL you pass to the Durable Object (or Service Binding) is 'meaningless' - it does need to be a valid URL, but otherwise means nothing.

There is definitely a consistency to these cloudflare designs which is good

You could pass

http://lol.bar

or you can pass a URL w/ query strings if you prefer to use those rather than headers.

And yeah, I did notice the url is meaningless - the code inside the DO fetcher in the sample app is parsing the path, but it's just looking for string matches, it doesn't actually care about the url structure unless I choose to make it care

It historically accepted just a relative path, like

fetch('/counter')

- but now requires a full URL - even if the scheme/hostname is pretty much meaningless unless you choose to use it in your DO.

Nice! Have you checked out Cloudflare R2 for your storage layer? Similar to S3, but easier binding from workers/DOs, and you could potentially save $$$ on egress bandwidth https://www.cloudflare.com/products/r2/

I'm looking for advice on structure. I have roughly this json structure (there are more attributes that this but this has enough for evaluating the concept):

{
  "xactnUnixTime": "bigint",
  "otherTransactionAttribute": "string",
  "transactionArguments": {
    "transactionArgument": {
      "argIndex:": "integer",
      "argName": "string",
      "argValue": "string"
    }
  }
}

I'm envisioning each record will be an instance of a DO with a xactnId as the name. If I have 100,000 recs and I need to get all recs with a transactionArgument.argName equal to a specific string and transactionArgument.argValue equal to a specific string - is that something doable with DO with decent performance and without writing tortured code to do it (a loop is fine)? Is it a reasonable use case for DO? I know storing it in a database or a json file, it's very quick to come back.

qq: can you make backups of DO's?

not yet (other than by doing it yourself), but the team is thinking about it: https://discord.com/channels/595317990191398933/773219443911819284/1074752539788062860

how do I upvote the team thinking about it 😄

I know of it, but as the rest of our stack is in AWS it seemed easiest to stick with what I knew for now. I do like the idea of not having to hop between cloud providers for this whole system.

maybe I'm misunderstanding your use case, but DOs aren't indexed beyond name of the DO, then name of the storage keys within the DO. If you make 100,000 DOs and want to find which ones have data matching some arbitrary criteria, you'd need to iterate through 100,000 DOs and then each key within them. So you're definitely better off with a DO naming / storage key naming convention that lets you find what you're looking for, and doing writes with that in mind

Hmm - so working backwards I would need to have a DO called TransactionArguments with the following keys/values: -XactnId -argIndex -argName -argValue I could make the "name" of each one {XactnId}-{argIndex} if that helped. Let's say I have 100K DOs. Could I get a list of the XactnIds which matched certain argNames and certain argValues, with quick performance (say <1 second)? If so, could I then easily take that list of XactnIds and return every instance of a different set of 100K DOs where either the name or the value of a XactnId key (I could do whichever was better) matched one in the list?

How much load will there be? If the answer is "not much", you might consider a single DO with 100,000 storage key/value pairs or maybe some way of grouping them? That might allow you to build your own indexes to speed up access. You are allowed an unlimited number of storage key/value pairs but I've only used 1,000 max in a single DO so YMMV.

storage.list(options)

accepts a

prefix

option so you can get the storage key/value pairs that start with your Xactnld. It also has paging support.

Thanks for the observations. Some follow up questions -is there any documentation on building my own indexes? -will the prefix option allow me to find key value pairs that start with all Xactnids from the list I generate with my first query, as opposed to a single xactnid match? I appreciate all the suggestions from everyone. My general sense here is that neither KV or DO are designed for what I am trying to do even if it is theoretically possible to get it to work. I like the idea of the cloudflare workers, and the project I am working on has an existing cloudflare api implementation (no back end). So we're looking at whether we can expand it to some calls with a back end. I suspect if D1 were further along, that would be exactly what I needed.

Hello all, I'm using the jurisdiction feature to restrict DOs to the EU. At least I think I am :). Is there some way to see the effect of using this feature to verify that it's working correctly?

Send a request to a "what's my IP" service from the DO? 😅

This is what I do to get the current datacenter location from a DO:

js
export class DO {
  async fetch() {
    return new Response(
      ((
        await (await fetch("https://www.cloudflare.com/cdn-cgi/trace")).text()
      ).match(/^colo=(.+)/m) as string[])[1],
    );
  }
}

thank you! super useful.

Happy to help!

I remembered

request.cf

but I don't have access to the incoming request in the area of code I'm in, but this I can do from anywhere.

(for future searchers, there is also this: https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties)

I had a similar thought about D1. As for your follow up questions, it's hard for me to get the entire picture of what you are trying to do. The use of the word "query" is also giving me heartburn. But... my use of "index" is not in the true database sense but rather another key/value pair where the key is hardcoded to something like "indexeByXactnids" and the value is literally an object where the key is the Xactnid and the value is an array with all the keys for the matching items. You could get fancier and create your own b-tree indexes but that's probably too far. As for your second question, if you had a list of Xactnids, you could loop through them to fetch the matching key/value pairs. However, you might run into the 128MB limit depending upon your key distribution patterns. My suggestion is to play around with it. Write some code. Do some experiments.

I appreciate response. A "query" is simply an inquiry but I take your point. What you suggest is already what I was planning, but with the need for searches nested three levels deep, that gets complex very quickly. And for me, hard typing of all my code (I write code generators to help) is something I won't compromise on unless I have to, so that piece would get more complex too. I'll revisit a cloudflare "back end" when D1 is at least well into beta. Note - I'm sure DO is wonderful for things like the messaging app which cloudflare has a sample of, or for very flat or very small data sets

It's good for more than the messaging app and the amount of data is not the limit so much as the amount of data you need for one operation. It's at the scale out end of the spectrum from scale up to scale out. Databases are more scale up. For instance, if you can hierarically determine that one user will only need this moderate amount of data for this one operation, it's ideal. But for things like analytics where you need to visit all the data to create an aggregation, it's not ideally suited.

Good summary. I'd also say not to expect D1 to perform meaningfully differently from Durable Objects for large aggregations. If you're used to e.g. columnar OLAP databases, that's not what this is.

That is an excellent summary, thanks. Almost everything I do is aggregation and analytics 🙂