This message was deleted.
# generals
Slackbot
12/04/2023, 5:57 PMThis message was deleted.
d
David K
12/04/2023, 6:14 PMYour app would create a subscription to each of the topics (events) it is interested in, consume the messages, and then populate an internal database based on these events to have its own “view” of the state of the world.
👀 1
h
Hoss
12/04/2023, 6:18 PMone common thing it seems I'd need to be able to do is pull all of the historic events for a given OrderId for example. Would I be pulling that from pulsar and the orderid is a common message key across the topics or something?
Hoss
12/04/2023, 6:23 PMI am a little surprised this is how it is implemented, I kinda expected to have one topic with a message key that varies by order ID. that way I could easily grab all of them
Hoss
12/04/2023, 9:46 PM@David K apologies for the ping and thanks for the help earlier, I was hoping you had more to say 😅
Hoss
12/04/2023, 9:46 PMi have watched multiple of your videos i think
thankyou 1
Hoss
12/04/2023, 9:47 PMi guess i'm trying to figure out if pulsar is a good place for an authoritative store of events for ES or not
d
David K
12/04/2023, 10:04 PMNo worries. It is possible to structure a topic with multiple event types in it and use the message key to filter them. However, that approach is generally considered an anti-pattern.
👀 1
David K
12/04/2023, 10:06 PMPulsar is a good store for events, given its long retention periods and support for schema evolution and millions of topics. But EDA application development will require consumption from multiple topics per service, which is sort of the whole purpose of the design, to allow each service to have its own view of the world.
David K
12/04/2023, 10:08 PMIf you place different messages inside the same topic, then the processing/scanning time of that topic will increase. Also, any problematic messages that cannot be processed will cause a “head-of-queue” blocking scenario that will delay the processing of other, unrelated events.
l
Loïc Berthou
12/04/2023, 10:18 PMInteresting… I was using EventStore in a previous company but we faced scaling issues
j
Julien Jakubowski
12/06/2023, 10:05 AM@Loïc Berthou I would love to know more!
l
Loïc Berthou
12/06/2023, 2:01 PMI wasn’t super close to the details, happy to provide some contacts if you’re researching the topic. From my recollection:
• Random dropped events when replaying streams. Happened rarely so made debugging even harder.
• Slowness of stream replays. Impossible to catch up in a reasonable timeframe when rebuilding our read models for large numbers of aggregates.
• Reliability issues on the server when put under heavy load.
• Difficulty in building composite streams.
I would say that we had a huuuuge EventStore database, which it was not designed for. We failed to archive and snapshot our streams before it became an issue.
But that’s also my biggest complaint with EventStore: I think it lacks a comprehensive library layer that helps with event sourcing best practices. We had to build it ourselves and made a few mistakes along the way before we learned how to do it properly.
👍 1
h
Hoss
12/06/2023, 8:34 PM@Loïc Berthou I was looking at EventStore too. Where did you end up, Pulsar? Can you say more about your ES setup there?
l
Loïc Berthou
12/06/2023, 8:41 PMWe looked at Pulsar at the time as a serious contender for a migration. We managed to solve most of the issues with EventStore or at least cope with them… I left the company since. I have some colleagues that are still exploring Event Sourcing with Pulsar (mostly for fun). Others are now still working with Event Sourcing but decided to go with MartenDb (I think addresses my complaints about having a simple-to-use library).
Loïc Berthou
12/06/2023, 8:42 PMTo be fair, I still think that EventStore was a great product and would probably fit most use cases. They are also continuously improving their product and since they’ve been working on it for a long time, it’s probably the most mature & feature complete.
Loïc Berthou
12/06/2023, 8:44 PMI would choose MartenDb for a small project because I’ve personally never used it (and want to experiment), and because it seems much simpler to operate (I’m not sure there is a “simple” way to run EventStore, but again it’s been a while).
🙌 1