Hi, is it still not possible to subscribe to a top...
# generalt
Tom Korn
04/10/2024, 2:14 PMHi, is it still not possible to subscribe to a topic with a kind of filter? That the consumer only receives messages with a given header or key?
y
Yu Wei Sung
04/10/2024, 2:44 PMis this helpful?
d
David K
04/10/2024, 2:51 PMYes, that is true for Apache Pulsar. However, StreamNative has developed pfSQL (proprietary), which does message-level filtering of messages based on the message content, keys, etc.
a
Alexander Preuß
04/10/2024, 4:49 PMDepending on what you want to do exactly, maybe writing a small Pulsar Function is the way to go here
a
Asaf Mesika
04/16/2024, 6:45 PMPulsar have added server side filtering
Asaf Mesika
04/16/2024, 6:45 PM@Enrico Olivelli you reviewed it right ?
e
Enrico Olivelli
04/16/2024, 6:52 PMI contributed it 😉
You can find a good example looking for Pulsar JMS, there you can find how to use JMS compliant server side filters
a
Asaf Mesika
04/16/2024, 6:55 PMDid you document it @Enrico Olivelli ? It’s a super strong feature
e
Enrico Olivelli
04/16/2024, 8:34 PMI am AFK, please search for Startlight for JMS on Google
a
Asaf Mesika
04/16/2024, 8:41 PMI mean it’s a pulsar feature so in pulsar docs no ?
e
Enrico Olivelli
04/17/2024, 7:22 AMUnfortunately I cannot find any docs on the Pulsar website 😞
Enrico Olivelli
04/17/2024, 7:23 AMit is a matter of:
Enrico Olivelli
04/17/2024, 7:25 AM• setting these settings in broker.conf
entryFilterNames=list-of-filters
eentryFiltersDirectory=/pulsar/filters
• have your EntryFilter implementation
That said, implementing filters is pretty hard because in the Pulsar Broker you really must try to not do any processing on the messages, in order to not impact latency,
also when coding an EntryFilter you must deal with the internals of Pulsar and this is not straightfoward
This is why I referred to JMS filters
https://docs.datastax.com/en/streaming/starlight-for-jms/3.2/examples/pulsar-jms-server-side-filters.html
because it is pretty straighforward to use and you really don't need JMS at all,
the filter works even without the JMS client, you just have to set the "selector" (that is a n expression like foo='bar') on a Subscription and then it works
👍 4
a
Asaf Mesika
04/17/2024, 12:57 PMI think since it’s such a good feature it’s worth describing it in the docs even if it’s short. Especially what was described here.
I think the most notable note to write in the docs is the way it works. Technically you need to hook the subscription into the filter and that is done using property or something which i don’t recall excactly
➕ 1
e
Enrico Olivelli
04/17/2024, 3:01 PMI am sorry but I don't have time these days 😞
👍 1
3 Views