Hello Team, I am working on a usecase where data i...
# troubleshooting
s
Hello Team, I am working on a usecase where data is being aggregated for a given window and then published to a sink. So this may not be a keyed window aggregation and I see windowsAll executes with just 1 parallelism. Any suggestions on achieving non keyed windowing?
r
So you want to run aggregate operator (non-keyed) with more than one parallelism ?
s
Thats right! Just the aggregation without doing key by
r
One approach to parallelize this can be first doing keyed pre-aggregations and then finally do windowAll on these pre-aggregations, if single parallelism of windowAll is bottleneck for you.
s
How does that solve the problem with windowAll. My requirement is non keyed windowing. keyBy is expensive operation and results in shuffling. I want to avoid that.