Having a bookkeeper replicaCount at 4, what would ...
# general
g
Having a bookkeeper replicaCount at 4, what would be a most natural Ensemble-Qw-Qa setting, if I want to tolerate one of the replicas going down even for a considerable amount of time (node outage, etc.). I believe E should be less than replicaCount (to tolerate one replica going down), while best to have E=Qw (avoid issues with lack of consistency). So I guess 3-3-3 or 3-3-2 would be best. What configurations would be recommended?
h
We recommend 2-2-2
2-2-2 can tolerate one data copy lost, and 3-3-3 can tolerate two data copies lost
g
I guess it's the reverse, 2-2-2 can tolerate 2 data copy lost?
d
For Apache BookKeeper, the most common and recommended quorum configuration for balancing latency, durability, and availability is typically
E=3, Qw=3, Qa=2
. This configuration provides a good balance between durability and latency. Every entry is replicated to all three bookies in the ensemble. This maximizes durability, ensuring each entry has a full set of replicas. The client waits for an acknowledgment from only a majority of bookies (two out of three). This lowers write latency by not waiting for the slowest bookie. Furthermore, the system can tolerate the failure of
Qa - 1
bookies without any data loss.
If you want to use an ensemble size of 4, then
E=4, Qw=3, and Qa=3
would provide additional durability and recoverability while avoiding the latency penalty of a slow bookie. The most detailed explanation of the BookKeeper storage system I have encountered is available here, and it is definitely worth reading if you want to better understand the tradeoffs involved with these decisions.
h
2-2-2 means write two copies, and make sure the 2 copies are written to two bookies before acknowledging to the producer. You only write 2 data copies, and can only tolerate 1 data copy lost
g
Thanks for all the responses!