pulsar

I’m going to start the release process of candidate 2.

<@U0152Q766AE> merged &amp; cherry-picked

OK. The RC2 is building. Let’s include it in the RC3.

The Pulsar 3.0.0 release candidate 2 is out now: <https://lists.apache.org/list.html?dev@pulsar.apache.org> Please help verify.:slightly_smiling_face:

We shouldn’t delay the RC too much because of some improvement issues or PRs.  Otherwise, it will cause people not to have enough time to verify it, which will also bring risks.

Yes. The RC3 will be built on April 25. It is the last RC in our plan unless there are any new blockers.

These PR's are important to PIP-195 and hopefully will make it into 3.0

Let’s raise the discussion on the mailing list to reach a consensus.

One additional point: if we don’t pass the `docker.platforms` arg, the image will be built without buildx, in the current host platform, 100% equal to today’s 

<@U680ZCXA5>. Thanks for pointing out the problem. I’m +1 for cherry-picking this PR.

If you're confident that the new pocess works, I'm also +1 as more and more people ask for ARM64 and it's nice to have in the LTS. And there's not real changes on the code itself so no risk of regression, only small risk of delayed releasing because of build issues.

I have tested it locally on branch-3.0. And it works fine for me: <https://hub.docker.com/repository/docker/snzkyang/pulsar/tags?page=1&amp;ordering=last_updated>

It’s a critical bug fix for the newly added feature. I’m going to cherry-pick it. /cc <@UDZPKSJSK> <@U03UNGJAKAM>

Sure. Are you reverting the commits mentioned in the email?

Yes except <https://github.com/apache/pulsar/commit/e248e14473142765db1df324c20a081a2422980e>

I commented on the commit I didn't revert: <https://github.com/apache/pulsar/pull/20155#issuecomment-1519738507>

Good finding. I think we should keep this one.

Sure. I’m OK with that. What is the chip architecture of your computer? Please note that we need to build the image on the intel machine.

But IIUC, we'll now cross-build with buildx for both archs.

(once we cherry-pick <https://github.com/apache/pulsar/pull/19432>)

The plan is to release a new RC tomorrow. Do you think we'll have a BK release with the fix by then ?
If not, I guess we'll do the RC as planned and will integrate the fix in the (hopefully) final RC of next week.

<@UDZPKSJSK> The rc-3 should be “last” rc :slightly_smiling_face:, otherwise the “announce” date cannot be fixed on 05/02

Then I guess we need to wait for 0 blockers

my 2cent suggestion would be:
• let’s wait tomorrow to understand how serious this issue is
• if it’s not too bad, continue with 4.16.1
• if it is, then defer RC3 by ~2 days for 4.16.2. We should still have time for 3days vote of 3.0.0, blog/release-notes/site/…. 

I discovered that the add/read request latency metric disappeared issue is caused by <https://github.com/apache/bookkeeper/pull/3837>. I think we do not need to trigger a new release.

The root cause is that <https://github.com/apache/bookkeeper/pull/3837> introduced the group flush read/write response to the client, and it doesn’t record the request level latency when flushing the responses to the Netty channel.

Impact range:
 - V2 protocol
 - bookkeeper_server_ADD_ENTRY_REQUEST
 - bookkeeper_server_READ_ENTRY_REQUEST

We can use `bookkeeper_server_ADD_ENTRY` and `bookkeeper_server_READ_ENTRY` instead to record the read/write latency. The difference between those two types of stats is:
 - `bookkeeper_server_ADD_ENTRY_REQUEST` or `bookkeeper_server_READ_ENTRY_REQUEST` records three time ranges: Request waiting in the `writeThreadPool`/`readThreadPool` + Requests processing time + Response waiting to write into the Netty channel
 - `bookkeeper_server_ADD_ENTRY` or `bookkeeper_server_READ_ENTRY` only includes the Request processing time

The two queued time cost is small and has a limited impact on the whole time, so I think we can add a note on the BookKeeper 4.16.0 release note and do not need to trigger a new release.

<@U680ZCXA5> Do you have any ideas?

For me I’d say to continue without a new release:
1. There is no impact on data plane
2. There is an alternative metric as a workaround
3. There’s not enough time if we do 72h for BK release + 72h for Pulsar release.
4. We can include in 3.0.1 shortly after

Sure, it works for me /cc <@U0152Q766AE> <@U03UNGJAKAM> <@UDZPKSJSK>

I'm starting the release process for the RC3

It happens during the push. Build and tag are OK.

I have encountered a similar issue before. But in my case, it seems to be just a network problem.

It was disk space :slightly_smiling_face:. Not very helpful error message...

It took me total 2 hours and 42 minutes to build these two images.

The grpcio pulled is a very old one (1.27). We should try to upgrade it to a version that has pre-built wheels for ARM.

As I understand it, it needs to match the one from BK so it would need to be bumped in BK first.

Actually the constraint has been removed on BK: <https://github.com/apache/bookkeeper/pull/3875>

It's been done. Just not in a pulsar-client-python release :cry:: <https://github.com/apache/pulsar-client-python/pull/111/files>

Yes, cross-compiling in Docker is very slow. It was much faster to build the double-image on Mac M1 :slightly_smiling_face:

Yes, it’s bigger because we are including GCC to build the Grpc wheel..

(which could be done in a separate Docker stage…)

HI <@UDZPKSJSK>, we need to trigger RC4 due to this breaking change: <https://lists.apache.org/thread/klomjqv40ybnnr4vvj9ljoktkxct6rrv>

I’m starting to collect and write the release note. /cc <@UDZPKSJSK> <@U03UNGJAKAM>

Hi, <@UDZPKSJSK> I think we can close the vote.

Also give me some minutes. I want to add my +1:slightly_smiling_face:

We have some more PMCs that are finishing their testing.

Yes. Enrico and Nicolo will soon cast their votes.

I’m going to update the pulsar-site according to <https://pulsar.apache.org/contribute/release-process/#update-the-document>. Can you help close the vote and promote the release? <@UDZPKSJSK>

Hi, <@U680ZCXA5> I will start to handle the rest of the finalized tasks later to make it hit the 05-02 deadline.

We should try to hit the 05-02 deadline :slightly_smiling_face:

The Pulsar 3.0.0 has been announced! <https://lists.apache.org/thread/w3gh6qpokl4h2d1qv2yr74xorsmnvsf1> Thank you all for making this possible!