Interesting talk by <@U08JY1C8MLN> about Pinterest...
# generalFelix GV
05/09/2025, 7:39 PMAt , it seems that the system implements the Hybrid Store design pattern. Is that the right understanding, @Jia?
m
Manu
05/09/2025, 7:45 PMNice. Looks similar to Venice on the interface/functionality layer with some distinctions (schema-less, c++ etc)
f
Felix GV
05/09/2025, 7:52 PMYep, many similarities. Even the scale numbers look similar 😄 …
k
Koorous Vargha
05/09/2025, 9:10 PMJust gave the talk a watch. It was very interesting to see the similarities (RocksDB, Helix, etc.), and how KVStore offers local embedding of the data too (DaVinci).
I see that the servers bootstrap batch data from S3. For the average use-case, how long does it take for a new version to come online, and does using this approach help your system be more resilient to rebalances?
f
Felix GV
05/09/2025, 9:28 PMThe Da Vinci / embedded part was in the future work section (and the server query pushdown as well), IIUC. Unless I misunderstood.
k
Koorous Vargha
05/09/2025, 9:29 PMYou're right. I misinterpreted.
j
Jia
05/12/2025, 11:12 PMHello @Felix GV, thanks for sharing.
At , it seems that the system implements the Hybrid Store design pattern. Is that the right understanding, @Jia?Yes, it's a similar pattern, and lots of our ads use cases (e.g. ads index) use this approach.
Jia
05/12/2025, 11:16 PM> I see that the servers bootstrap batch data from S3. For the average use-case, how long does it take for a new version to come online, and does using this approach help your system be more resilient to rebalances? (edited)
@Koorous Vargha, we do that pretty efficiently today, with the EC2 instance type we have, we are using about 50% of network bandwidth for that, averaging 200MB/s per VM, and it scale with the number of VMs we have.
And it comes with challenges as well, as we run into frequent network throttling issues as discussed in the talk.
f
Felix GV
05/13/2025, 10:48 AMThanks for chiming in! And congrats again on the talk!
❤️ 1
2 Views