L P V
06/24/2024, 1:54 PMDataStream<Event> eventDataStream = env.fromSource(
eventKafkaSource, strategy, "TrackingEvent Source");
DataStream<Event> filter = eventDataStream
.filter()
.filter()
.setParallelism(8)
.partitionCustom(new CustomPartitioner(), (KeySelector<Event, String>) Event::getId);
DataStream<MultiFeatureStreamingMessage> featureStream = filter
.keyBy()
.window(SlidingProcessingTimeWindows.of(Duration.ofMinutes(60), Duration.ofMinutes(5)))
.allowedLateness(Duration.ofSeconds(60))
.process(new Processor())
.setParallelism(20);
Because when I monitoring job on Fink UI, the featureStream have 10/20 task slot always busy with 100%. But I think it should not happen when I've already partitionCustom for filter?
Or I have add partitionCustom for both filter and featureStream datastream?