https://discord.cloudflare.com logo
Join Discord
Powered by
# durable-objects
  • g
    thats why they partnered with macrometa
  • g
    because durable objects are meant to coordinate state and maintain state with locality, but not meant to provide a full database suite of features.
  • w
    Well... thats was my question, if i can access the DO storage without load the DO i can do my search.
  • g
    no because its stored in cold storage until its accessed, and then its kept in memory until its evicted
  • g
    if they allowed everyone to keep their durable objects in memory all the time, they would run out of memory on their edge servers
  • g
    but you cannot access DOs from the disk/cold storage directly
  • g
    Trust me, its a feature I want also, but there is hope. I am prototyping a way to create indices with durable objects to index objects and search them efficiently.
  • g
    however, this wont spin them up to search them, because there would be an inefficiency in that. Not to mention the cost for potentially millions of reads every time you perform a search
  • w
    Im not talking the memory.... Im talking about the storage... Thats not much different the current KV! In currently i can search all my KV namespaces.
  • g
    yes
  • g
    because KV is designed for read heavy workloads
  • g
    KV is bad for concurrent/highly consistent write workloads
  • g
    hence why DOs were created imo. To coordinate state amongst write heavy workloads
  • w
    Im already have a production software that i have a 2GB database using durable objects, works fine because my values is inside of 32kb limits.
  • w
    You are talking a problem that i did not have using DO
  • g
    you are going to have a problem searching them
  • g
    as you are asking about it now, here.
  • g
    and I am telling you that there is currently no way to access their storage without spinning them up, and probably never will be.
  • w
    My problem is if i want to search then if a split my bigger DO in smaller ones
  • g
    search gets super complex really quickly and durable objects weren't really build around indexing
  • g
    then just split it and do binary searching on the list of them
  • w
    As keanton said: "If you're getting near the limit of what one Durable Object can handle, you probably should introduce such sharding into your app. Once you have sharding, you can scale easily again, by adding shards."
  • g
    yeah
  • g
    because durable objects are single-threaded, their maximum throughput is basically determined by the throughput of the machine running them (a single thread of that machine)
  • w
    Never will be? Thats response is a official cloudflare statement?
  • g
    no, thats my speculation.
  • g
    Durable objects aren't really built for that
  • g
    because when you try to access a durable object's cold storage, without spinning it up, you would experience quite a bit of latency and the infrastructure wasn't really designed to query cold storage.
  • g
    its why durable objects took the form they did. You can quickly throw write-heavy workloads at them and they can sit there in memory coordinating the state and resolving conflicts
  • g
    but if you have them in cold storage, you have no way for conflict resolution to happen
1...112113114...567Latest