Hi all, I have a DataStream which partitionCustom,...
# troubleshooting
l
Hi all, I have a DataStream which partitionCustom, follow is a windowSliding. Should I rebalance again before windowSliding
Copy code
DataStream<Event> eventDataStream = env.fromSource(
eventKafkaSource, strategy, "TrackingEvent Source");

DataStream<Event> filter = eventDataStream
.filter()
.filter()
.setParallelism(8)
.partitionCustom(new CustomPartitioner(), (KeySelector<Event, String>) Event::getId);

DataStream<MultiFeatureStreamingMessage> featureStream = filter
.keyBy()
.window(SlidingProcessingTimeWindows.of(Duration.ofMinutes(60), Duration.ofMinutes(5)))
.allowedLateness(Duration.ofSeconds(60))
.process(new Processor())
.setParallelism(20);
Because when I monitoring job on Fink UI, the
featureStream
have 10/20 task slot always busy with 100%. But I think it should not happen when I've already partitionCustom for filter? Or I have add partitionCustom for both filter and featureStream datastream?