pulsar

Trying to run some basic integration tests pointing to a docker-compose setup and the CPU is constantly really high:

This is what I see when I run `top` inside the container

This is what a scale up from 200 to 400 consumers looks like

I gradually expanded from the initial 200 consumers to 600 consumers, still can't reproduce the problem.

I was assuming this would be enough to enable it as I don't see any documents indicating otherwise. Perhaps I didn't restart the (correct) relevant scripts?

Is there a way to define message properties inside a Typed Pulsar Function ?

Hello, is it possibile to configure a connector using json instead of yaml? There is an hint here: <https://pulsar.apache.org/docs/en/io-quickstart/> . but when I try to create a sink with the Admin REST API I get this error “Unsupported Media Type”

The command "mvn clean install -DskipTests -rf :pulsar-kafka-compat-client-test" was executed, and BUILD FAILURE was shown. The info for the error is :
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project pulsar-kafka-compat-client-test: Compilation failure
[ERROR] /mnt/myshare/myGitLocal/pulsar/tests/pulsar-kafka-compat-client-test/src/test/java/org/apache/pulsar/tests/integration/compat/kafka/PulsarKafkaProducerThreadSafeTest.java:[24,41] 找不到符号

Here's the output of `pulsar-admin topics stats-internal` on the topic.

Yep with 2Gi it's ok but still a realy big amount of cache usage in the container :

Hi, I encounter the 500 HTTP internal server error when using pulsar-admin on a specific topic after publishing messages into it for days. How can I debug this?

Hey, I’ve faced weird classpath issue when running in cluster mode,  so I prepared a little project to show an issue
Pulsar can’t find my SerDe implementation, even its same package as the function (and it finds the function without issues)

<https://gist.github.com/lanwen/769a1204b26f8392c3929c7457a980e2>

project is here <https://github.com/lanwen/pulsar-functions-example>
and the test class
<https://github.com/lanwen/pulsar-functions-example/blob/master/tests/src/test/java/ru/lanwen/pulsar/functions/IntegrationTest.java#L132>

starting this test its expected to see both input and output, but only that exception is thrown. Other modes with another 3 methods work fine. Maybe you would have some thoughts?

So it somehow can’t be loaded by classloader

This is what I have in my standalone.conf w.r.t to TLS and ports, etc.

here is the standalone.conf that I am using

These settings work for me. I did notice in your last stack trace that the TLS handshake was failing on port 6650, and not 6651. My guess is that if you change the corresponding config to use the proper port things should work.

I have one subscription on this partition topic and it  seems normal.

How could this happened? The only thing I noticed that is the cursors of subscription on different partitions are quite different:

The cursor on the 6GB partition:

Thank you sijieg, here is the stats of one 6GB partition:

After restarting my client, the unbalanced situation has gone.

Unfortunately it has more dependencies like protobuf. I’m using client-all and pulsar-api and still getting this:

The updated values.yaml file that I am using

Hi <@U6E5YMRKR>,

As mentioned above, even without changing any properties on keepAlive (both and broker and client side), connections get closed after 60 seconds and a new connection is established automatically and it is used for futher produce/consume messages. Is there an option to keep the connections alive for a given time period, as in JMS?
(I see this happening only when i pause above program with debug mode). Attaching the program used and log here

<@U680ZCXA5> Thanks for looking into this.  Here are the broker logs during startup.

Seems changed to Waiting: PodInitializing rather than the red warning