From my POV, YugabyteDB is a perfect
# spicedb
a
From my POV, YugabyteDB is a perfect alternative to CockroachDB for SpiceDB. However, its default behavior conflicts with strict Zanzibar requirements. I want to build a custom Yugabyte datastore driver (forking the native Postgres driver as a base). Here is my breakdown of the challenges and how I plan to tackle them: __The Problems__ **Isolation Mismatch**: Yugabyte defaults to Snapshot Isolation. SpiceDB strictly needs global SERIALIZABLE to prevent the New-Enemy Problem. **Optimistic Concurrency Overdrive**: In Serializable mode, Yugabyte resolves conflicts via Fail-on-Conflict. High SpiceDB write/read contention will cause massive 40001 (Serialization Failure) aborts, destroying throughput. **Clock Skew Sensitivity**: Without TrueTime hardware, Yugabyte uses HLCs. If node clocks drift, multi-row linearizability risks breaking. __**The Implementation Ideas**__ **Enforce SERIALIZABLE per connection**: Explicitly enforce it upon session initialization (SET SESSION...) instead of relying on global DB configs. **Driver-Level Retry Engine**: Intercept 40001 errors and resolve them internally via Exponential Backoff + Jitter to keep worker nodes from smashing into each other. **Pessimistic Locking via FOR UPDATE NOWAIT**: For high-contention rows (namespace/relation updates), force Yugabyte to fail immediately so the driver's retry engine can back off instantly rather than blocking connections. **Time-Travel Queries (AS OF SYSTEM TIME)**: Map SpiceDB’s ZedTokens directly to Yugabyte’s historical MVCC reads. Since historical reads don't block writes, this should eliminate a massive chunk of read/write conflicts. **Hardened Cluster Requirements**: Document that deployment requires aggressive NTP/Chrony sync (clock skew < 5ms) and a hardened --max_clock_skew_usec flag. __Torture Me / Feedback Requested!__ I'm pretty stubborn, and even though I don't fully grasp every SpiceDB edge case yet, I'm going to see this through. Please hit me with honest feedback!
j
I'd take a look and how the strict serialization is handled in the CRDB driver: its only enforced if you choose to do so, often on a per-tenant basis, to reduce the contention
y
i'd also look at the changes that we recently made to unify schema storage - if i were implementing a new datastore driver today, i wouldn't bother implementing the old row-per-schema-entity logic
a
Good point, but there is a major architectural difference between CRDB and Yugabyte here: As I understood CockroachDB is always SERIALIZABLE under the hood; you cannot lower its engine isolation level. So even if the CRDB driver loosens its checks per tenant, the underlying DB still guarantees safety against Write Skew. YugabyteDB, however, and that was one of the github issue points, natively supports both SNAPSHOT (as default) and SERIALIZABLE. If a user connects to Yugabyte and runs SpiceDB on the default, SNAPSHOT isolation, the Zanzibar guarantees break globally due to potential Write Skew. That’s why I wanted to enforce it via the driver session - not trusting that somebody configures yugbyte always correctly - ( SERIALIZABLE on) - and so protecting the user from accidentally breaking the core Zanzibar model. Does that make sense to you?
j
Yes but you ideally want it per tenant or something similar or it will cause too much write contention
a
What do you mean "per tenant" ?
Update: - I took the postgresql datastore, copied it, - adjusted the code (to the best of my knowledge and belief) for yugabytedb, - created a new custom Docker Compose yaml which uses yugabytedb and got it running. - Can create a basic schema, can save a relation and can check the permission (yes/no) (tested via curl) Todo: - I need to create a pr. - I need to test it further and in greater detail. - Going again through the new code to find dead code other problems etc. Question: - What tests do you have in mind that should / can be tested. --- See ya
j
we have a ful datastore test suite
you should be able to run it over the impl
30 Views