This message was deleted.
# generals
Slackbot
01/31/2023, 11:42 PMThis message was deleted.
🙌 1
z
Zac Bentley
02/01/2023, 2:51 PMYes, we are overriding the name.
How does a producer's decision to connect to a single partition affect the consumers' ability to use KeyShared semantics?
Zac Bentley
02/01/2023, 2:52 PMThat is: our consumers are connecting normally, without overriding the partition (for now, we may change this for the same reason we changed the producer to do this: client instability and bug rates relating to partitionedness, errors, and producer creation failures). Only our producers are doing partition selection (and round-robin/key-based partition selection etc) at the application layer.
d
David K
02/01/2023, 3:54 PMSince your consumers are attaching to the partitioned topic using the key-shared subscription, they will be assigned a hash-range based on the number of consumers on that subscription. Normally, all of the producers are using key-based batching and routing and therefore we know that messages with the same key are all getting routed to the same partition. This was made the default behavior to make the consumption side more efficient and, more importantly provide global total ordering of the messages on a per-key basis. With your approach, messages with the same key are now spread across multiple topics which breaks the total ordering guarantee.
z
Zac Bentley
04/16/2023, 1:32 PMWe additionally hash based on partition count before selecting a partition to "force"-select. Everything seems to work well so far.
Zac Bentley
04/16/2023, 1:33 PMBasically, we do more or less what the client does (except we know the partition count a priori and don't continually poll for it, as it's only changed during Pulsar schema migrations): we hash over the partition count, open a producer on a given partition if one doesn't already exist, and send the message (including the key in the metadata) via that.
We find that this avoids a large number of bugs and serious quality issues in the C++/Python Pulsar Client.