It’s not this, it’s that you’d accumulate terabytes of WAL before any cleanup happens

In my use case, WAL files are often 300-500MB each

And one terabyte at this price is already 150USD a month, basically egregious

@ReubenBond the GC is pretty conservative right now.. It only removes stuff older than like 24h or so. It can be tuned much shorter, but we have an outstanding issue we need to fix in 0.9: https://github.com/slatedb/slatedb/issues/604 Until that issue is fixed (depends on Sujeet's compactor work first), you have to keep the GC pretty long

Annoying, but it'll get fixed soon enough

On a side note, would someone cost conscious enough to try to save a couple dollars a month that way be using azure or aws in the first place? 😝

Saving a couple of dollars by using slower storage, that is?

Not sure to get your point, did you benchmark both? In my own testing I didn’t really see any noticeable difference at all with slatedb

I don't quite understand what you meant about saving $

You brought up that premium could be cheaper than hot

Oh, I meant more that it's not necessarily more expensive - it depends on your storage size vs API calls

Yeah I got that 👍

If you want much lower latency (nice for a WAL), then you can get it and you don't necessarily have to pay more. It depends on a variety of factors though. In the current version of slatedb with the current GC, it seems like you'd likely pay significantly more for the latency reduction, but with some changes it might become an attractive option

I didn’t really see much difference in my own testing latency wise

And Microsoft doesn’t seem to guarantee anything either

Perhaps it’s there, in some workloads, or that it’s more consistent, or something but their sales pitch only states “premium is ssds”, premium is not even mentioned there https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview

I'm sure some PM is to blame. Premium is an account type, not an access tier

Funnily enough, they claim that hot is lower latency than premium > The hot tier has the highest storage costs, but the lowest access costs Ahaha

within a given account type (the account type would be Standard, IIRC)

Got it 🙂

I am pretty surprised that they expose that to the user in that way, I don’t dislike it but it’s pretty different from what others are doing

If they want to drive adoption, it's not the best way about it

When I was skimming through the docs it seems that in the past a storage account could be more or less tied to a storage product, that kind of seems to ressemble the premium stuff

There is also Ultra Disk, which is different again. They appear in the Socrates paper (https://www.microsoft.com/en-us/research/wp-content/uploads/2019/05/socrates.pdf)

The cloud is here to stay. Most start-ups are cloud-native. Furthermore, many large enterprises are moving their data and workloads into the cloud. The main reasons to move into the cloud are security, time-to-market, and a more flexi- ble “pay-as-you-go” cost model which avoids overpaying for under-utilized machines.

😵‍💫 🥲

I guess this didn’t go through any external peer review

https://www.stepsecurity.io/blog/ctrl-tinycolor-and-40-npm-packages-compromised hmm again

It was in sigmod, so I'm sure it went through some peer review

A proper peer review would arguably never allow that intro 😅