Hello, is anyone familiar with how exactly the ope...
# troubleshooting
v
Hello, is anyone familiar with how exactly the operator chaining affects the data propagation mechanisms? We're facing severe performance issues after adding a number of
KeyedCoProcessFunction
s into the pipeline. As far as I can see, these operators are currently not chainable. Subsequently, they are forced to go through a "network" serialisation barrier here, despite having exactly the same parallelism. I find it a bit unexpected and wonder if there is a way to avoid it. Example job graph:
Copy code
Source A -> keyBy(A) --+-> KeyedCoProcess -> Sink
                      /
Source B -> keyBy(B) +
If I have either
Source A -> keyBy(A) -> KeyedProcess -> Sink
or
Source B -> keyBy(B) -> KeyedProcess -> Sink
independently, then everything after
keyBy
is chained as expected and the information is passed without extra serialisation.