In Apache Pulsar, when a message is successfully written to 2 out of 3 bookies, the process responsible for ensuring the message is replicated to the third bookie is called the "rereplication process." This process is part of the autorecovery mechanism in Apache BookKeeper, which is used by Pulsar for persistent storage.
According to the
Apache BookKeeper documentation, the rereplication process involves the client going through all ledger fragments in the ledger, selecting those that contain the failed bookie. A recovery process is initiated for each ledger fragment, and the client selects a bookie to which all entries in the ledger fragment will be replicated. The client reads entries from other bookies in the ensemble and writes them to the selected bookie. Once all entries have been replicated, the ZooKeeper metadata for the fragment is updated to reflect the new ensemble, and the fragment is marked as fully replicated.
Would you like more details on how this process works?
Sources:
-
Apache BookKeeper documentation 👍 0 button 👎 0 button 💡
Hint: Mention @runllm in the thread for followups.