Hi Team, We are currently operating a clustered en...
# general
p
Hi Team, We are currently operating a clustered environment consisting of two instances, running Apache Pulsar version 4.1.1 . During load testing, with "brokerDeduplicationEnabled= true", we have encountered issues related to message loss. For your reference and to support further analysis, I have attached the broker configuration file. And the MessageId corresponding to an unpublished message is. { "partition": -1, "ledgerId": -9223372036854800000.0, "batchIndex": -1, "entryId": -9223372036854800000.0 } DotPulsar version 4.3.2 is being used as the client library. We would appreciate your assistance in investigating this issue and welcome any insights or recommendations you may have.
l
If you have a way to isolate the issue with a exact instructions to reproduce it, it would be a valuable contribution towards solving your issue. It could be the easiest to target reproducing just with Pulsar standalone running in docker and using a DotPulsar app that reproduces the issue.
During load testing, with "brokerDeduplicationEnabled= true", we have encountered issues related to message loss.
Are you setting a unique producer name for each of your producers? That's necessary when using broker deduplication.
p
@Lari Hotari The issue can be reproduceable with Pulsar standalone instance running in docker and using a DotPulsar app. > Are you setting a unique producer name for each of your producers? Yes, we are using unique producer name for each of your producers
l
> The issue can be reproduceable with Pulsar standalone instance running in docker and using a DotPulsar app @Praveen Gopalan Since https://github.com/apache/pulsar-dotpulsar/issues/277 has already been reported about this issue, please share the (minimal) reproduceable app in that issue with steps to reproduce that can be followed. You can share the app in a separate GitHub repository, a forked repository or with files in a GitHub gist. The next step could be to have a failing test case for DotPulsar and then finally fixing the issue by bridging the gap and making the test pass.