So my initial take was it seemed very smoke and mirrory.

Like.... The claim is cost reduction. EXCEPT it seems for serving the data sacrifices are gonna have to be made

Cost is always kind of tenuous with anything in the cloud… like… the value prop of the cloud is elasticity and ease of getting started. Cost is not good with the cloud.

But… maybe some niche workloads that are either very elastic or where a large % of data is cold, then maybe S3 is the killer building block for those even in terms of cost

Training fits that description

But I think the main value prop is not cost… it’s operability, scalability…

That sounds right. Espresso has had bootstraps off of NFS for a decade+. But all the plumbing isn't trivial

Ultimately, you still need to keep a copy of the data warm, so it’s sort of smoke and mirrors… but even that may be changing with stuff like S3 Express One or whatever tf it’s called

Very true, At Splunk we did exactly this. Moved all the state to S3 to make the stateful search indexes stateless but merely a cache. But the cache warming for latency impact was extremely tedious and did not get right till I left 🤷

Went to TiDB conference last month, and they have a product called TiDB X which does this: 1. S3 as blob store for durability 2. Copy of data in local SSD for fast access 3. Flush infrequently to S3 4. Raft log is the WAL that guarantees non-backed up data is still durable

Main value prop was elastic scalability

Everything but step 4 is espresso (and swap S3 for HDFS)

It seems that Google does things differently, though. They don't seem to use local SSDs much. All on remote block storage

For what is worth, at Astra DB we do the same with Cassandra: keep the data on S3/Google cloud storage/Azure Blob storage....then cache to "local" (to the network) SSDs in order to still get great performance for reads. Cassandra has immutable SSTable files, so caching is "doable" Caching is not easy, but you can get great performance if you do it well 😉

Wow… TIL about this. Really makes my mind do acrobatics…! https://github.com/tansu-io/tansu

With LinkedIn's move to Observe and Opentelemetry, did we ever pull out the backend to something like Prometheus? I'm asking because yesterday I found out how Histograms work in Prometheus and I'm horrified. It can get wildly misleading.

We keep hearing about systems that are "S3 compatible" or aspire to be… I thought nothing of it, until the other day someone shared this PDF of the S3 API… 😳 https://docs.aws.amazon.com/pdfs/AmazonS3/latest/API/s3-api.pdf

This is the first half of the doc 😱 …

Interesting, honest, take from the developer of Mongo’s causal consistency. Could be intetesting to build a causally consistent Venice FC+Producer combo client. Ideally with lessons learned from this post to make the API more intuitive 😅 … https://emptysqua.re/blog/how-to-use-mongodb-causal-consistency/

Interesting https://bsky.app/profile/aratno.bsky.social/post/3m5i2sne2x227

What Kuzu became after that startup closed down. Could be interesting. https://github.com/LadybugDB/ladybug

https://www.linkedin.com/posts/francoisp_smokedmeat-a-red-team-tool-to-hack-your-activity-7450167364517523457-d-A6 Might be worth taking a look if the Venice GHAs have any exploitable vulnerabilities!

This is quite interesting… Spanner as a docker you can deploy anywhere, with Colossus and TrueTime stubbed with non-Google alternative impls! https://cloud.google.com/blog/products/databases/introducing-spanner-omni

idk how future-tech this is. But I'm currently evaluating zk-esque technologies for building a very large cluster (50k+ nodes). I've got a list to get through, but I'm doing benchmarks on ZK and etcd. etcd is doing pretty dang good. 50k node cluster or bust!

Now I understand why I haven’t been getting value from LLMs… gotta try this next time! https://www.linkedin.com/posts/simonwardley_how-to-use-chatgpt-in-one-easy-lesson-ugcPost-7462811938192932864-kSHF?utm_source=share&utm_medium=member_ios&rcm=ACoAAAEk238BXlhz1s5hXc96bKIJJ-eWwXUnlas

I'm having an interesting realization. The way my company operates is we have an MCP server that we host. Customer agents interact with it in order to get information about clusters provisioned under their account. A cluster which gets provisioned has two visible steps. One is the 'provision substrate' step and the 'create cluster' step. The 'provision substrate' step is there originally because AWS EKS is super slow to provision (can be as long as 20 minutes). But as I'm staring at the architecture, I now realize this looks very similar to the colo build out.