I experienced a blip today where my DO (used for websockets) stopped responding to connections for ~2 minutes, and then seemed to come back online. Nothing seemed to show up in logs. Any advice on debugging this in the future? Does it sound like something within what I should expect as normal occasional maintenance?

That's certainly not supposed to happen, but in any big distributed system it's impossible to avoid the occasional hiccup. 2 minutes sounds a bit long to me though, I would have expected it to fail over sooner than that.

makes total sense that this would happen from time to time - i guess I just am curious if I can see what happened in a log somewhere? (for instance, not sure if it was the machine that failed over, or if it was something network related that shifted)

https://medium.com/geekculture/orchestrating-serverless-from-serverless-bcdb751ddd6c -> cool article on using xstate state machine library within DO. Always wondered how would the two play together. Maybe anyone used other means of implementing finite-state-machine pattern in DO?

Is there a foolproof way to detect a DO failure from a fetch? As in, I send a request and the isolate gets shut down while processing, for some reason or another. Can I catch that somehow?

Here's what I use:

typescript
import { DurableObjectStub } from './deps.ts';
import { sleep } from './util.ts';

export type RetryListener = (e: Error, retries: number, maxRetries: number) => Promise<void>;

export async function durableObjectFetchWithRetries(opts: { obj: DurableObjectStub, maxRetries?: number, listener?: RetryListener }, url: RequestInfo, init?: RequestInit): Promise<Response> {
    const { obj, listener } = opts;
    const maxRetries = opts.maxRetries || 3;

    let retries = 0;
    while (true) {
        try {
            if (retries > 0) {
                const waitMillis = retries * 1000;
                await sleep(waitMillis);
            }
            return await obj.fetch(url, init);
        } catch (e) {
            if (isRetryable(e)) {
                if (retries >= maxRetries) {
                    throw new Error(`fetchWithRetries: Out of retries (max=${maxRetries}): ${e.stack || e}`);
                }
                if (listener) {
                    await listener(e, retries, maxRetries);
                }
                retries++;
            } else {
                throw e;
            }
        }
    }
}

//

function isRetryable(e: Error): boolean {
    const error = `${e.stack || e}`;
    if (error.includes('Network connection lost')) return true; // Error: Network connection lost.
    return false;
}

Just throws a regular error, got it. Thanks!

How do we communicate between 2 DO instances (from potentially different classes) ?

Each DO instance is passed an env as the 2nd constructor arg that, just like workers, may have bindings to DOs. So they can make the same stub calls that non-DO workers do.

Thanks @User

I also got a architecture design question. I have a task queue organised in time bucket. Each bucket id is basicaly Math.floor(timestampInSeconds / (60 * 10)) so there is one bucket for every 10 min slot Now I am wondering if should have one global DO that store each bucket as key, or if I should have one DO instance per bucket. the way the system work is that a CRON job is executed every minute or so and read from the current and previous bucket for task to execute it will read and write to update the status of that task User will submit tasks that will be allocated into the buckets. so if I have a global DO, it will potentially have many writes against it, but keys are kind of isolated I assume so it should be fine ? Having one DO per bucket might have the disadvantage of id creation issue mentioned in the doc, where DO server need to agree on the canonical id. any suggestion ? is my analysis wrong ?

I had some issues with logs from Durable Objects not appearing in

wrangler tail

. Turns out making a

fetch()

request at any point while handling the request will cause logs to not appear...

js
export default {
    async fetch(req, env) {
        return env.LOGGER.get(env.LOGGER.newUniqueId()).fetch(req)
    }
}

export class Logger {
    async fetch(req) {
        const url = new URL(req.url)
        console.log(url.pathname)
        if (url.pathname === '/fetch') {
            // This causes logs to not appear
            await fetch('https://www.cloudflare.com/')
        }
        return new Response('Hello, World!')
    }
}

Does anyone know if using list() will count as a read for a all items in the list? Regarding pricing

If you will use list() and it includes 1000 items, will it count as 1000 reads?

Interesting, I can reproduce that as well with your exact code. I'm thinking it has something to do with an outstanding Promise, as awaiting the response body fixes it (console.log is seen in tail)

js
            const res = await fetch('https://www.cloudflare.com/')
            const txt = await res.text();

From cron, where you are not worried about being as close to multiple users as possible, if you are just using the DO for keeping state, I'm sure a single global DO would keep up with your reads/writes. Splitting the work into buckets might be useful for other reasons, like you can easily nuke all storage by bucket with .deleteAll afterwards - but of course it would make queries across buckets more involved. Naming shouldn't require coordination, could do something like

namespace.idFromName(new Date(timestampInMillis).toISOString().substring(0, 15))

to get a canonical name for a 10-minute period.

Pricing still doesn't seem to be 100% locked down yet, but I would assume that, like KV, List ops have their own pricing.

I think it was discussed here, and it might be based on the amount of data list will return (the 4 KB increments)

Is it possible to call a Durable Object from within a worker without exposing the endpoint? Or do you need to always add some sort of verification through an API key or something?

For example when setting up a CRON

Yeah, you can have a Worker with no Routes and the Workers.dev subdomain disabled. Doesn't have to have a durable object on it.

I have a worker with a CRON that needs to do some things inside the Durable Object. But since the Durable Objects only accepts fetch method, there is no way to call it internally right without exposing the DO to the internet?

DOs are never exposed to the open internet. The fetch method used to connect to a DO is tied to that DOs namespace, and thus can't be called from anywhere but Workers on your zone.

But it needs a request to do something

with fetch

It isn't the normal fetch, it is

MyDo.fetch

. Without access to

MyDo

, which no one else can get, they can't access your DO.

but what do I pass as first parameter if its only internally?

You pass a request. It is treated like a request to any other external request, but the method of accessing it is only available to you.

Example:

const id = env.DURABLE_OBJECT.idFromName('IDNAME')
const obj = env.DURABLE_OBJECT.get(id)
let resp = await obj.fetch('/internal-only')

just something like this?