Hi, I am considering using Apache Flink and, more ...
# troubleshooting
j
Hi, I am considering using Apache Flink and, more specifically, Flink Stateful Functions, to structure an indexing application. So far it seems to be an ideal architecture for the application. However, are a few concerns I need to address before making the final decision, which I haven't been able to clear out by reading the documentation and searching the web. I was hoping someone could please shed some light on them for me: 1. Is there a (standard) way to share state between (embedded/co-located/remote) statefun function instances? In my particular case, there would be many (million) stateful function instances whose processing depend on some global parameters that may eventually change. 2. Is there a way to broadcast (or multicast) messages to all (a group of) stateful functions? 3. Is there a way to access the stateful functions' states from OUTSIDE the stateful function themselves, e.g. by making some query against the whole set of instance states? 4. I am trying to decide whether to use Kafka or Pulsar for my streaming storage needs. Besides the pros and cons they have, when compared with each other, I've noticed that Flink's documentation seems to favor Kafka. Does that mean that Kafka's support in Flink is much more mature than Pulsar's?