Nathanael England
02/22/2023, 4:37 AMCoBroadcastProcessFunction and KeyedCoBroadcastProcessFunction?Martijn Visser
02/22/2023, 11:31 AMNathanael England
02/22/2023, 5:46 PMKeyedCoProcessFunction. Stream A is some rule information and stream B is the data the rule should be evaluated over. However, the timestamp information that flows in over stream B isn't the full picture so I wanted to attach a third stream of information that includes all the observed timestamps from the devices that make up stream B instead of just the limited amount of timestamp info that trickles in. I didn't want a global watermark strategy since there's no way to generate watermarks per source. I was originally going with the broadcast pattern for the time and let the process function downselect what it actually needed. I think that's fine for few devices but likely has issues as throughput goes up with more devices added. Without a broadcast pattern, I'll have to create a stream of timestamp information, duplicate the messages as needed and add a key so that it can be joined by union with stream B.Martijn Visser
02/22/2023, 5:48 PM