Travis sturzl
06/11/2024, 9:16 PMDaniel Połaczański
06/11/2024, 10:17 PMTravis sturzl
06/11/2024, 10:37 PMDaniel Połaczański
06/11/2024, 10:37 PMDaniel Połaczański
06/11/2024, 10:37 PMTravis sturzl
06/11/2024, 10:37 PMTravis sturzl
06/11/2024, 10:38 PMIt is important to recognize that you have no control over the order in which theandflatMap1callbacks are called. These two input streams are racing against each other, and the Flink runtime will do what it wants to regarding consuming events from one stream or the other. In cases where timing and/or ordering matter, you may find it necessary to buffer events in managed Flink state until your application is ready to process them. (Note: if you are truly desperate, it is possible to exert some limited control over the order in which a two-input operator consumes its inputs by using a custom Operator that implements the InputSelectableflatMap2
Daniel Połaczański
06/11/2024, 10:38 PMDaniel Połaczański
06/11/2024, 10:39 PMTravis sturzl
06/11/2024, 10:39 PMDaniel Połaczański
06/11/2024, 10:39 PMDaniel Połaczański
06/11/2024, 10:40 PMDaniel Połaczański
06/11/2024, 10:40 PMDaniel Połaczański
06/11/2024, 10:40 PMTravis sturzl
06/11/2024, 10:40 PMDaniel Połaczański
06/11/2024, 10:40 PMDaniel Połaczański
06/11/2024, 10:41 PMDaniel Połaczański
06/11/2024, 10:41 PMDaniel Połaczański
06/11/2024, 10:41 PMTravis sturzl
06/11/2024, 10:41 PMBoundedOutOfOrdernessWatermarks to try to keep things in order, but it doesn't seem to keep the events from 2 different streams in order with eachotherTravis sturzl
06/11/2024, 10:41 PMDaniel Połaczański
06/11/2024, 10:42 PMTravis sturzl
06/11/2024, 10:42 PMmap , I assume the data produced by the map inherits the watermark?Daniel Połaczański
06/11/2024, 10:43 PMWhen enabling the alignment, you need to tell Flink, which group should the source belong. You do that by providing a label (e.g. alignment-group-1) which bind together all sources that share it.Daniel Połaczański
06/11/2024, 10:44 PMDaniel Połaczański
06/11/2024, 10:44 PMIn order to achieve the alignment Flink will pause consuming from the source/task, which generated watermark that is too far into the future. In the meantime it will continue reading records from other sources/tasks which can move the combined watermark forward and that way unblock the faster one.Travis sturzl
06/11/2024, 10:48 PMTravis sturzl
06/11/2024, 10:48 PM