hey 👋 i have a few questions, with watermark alignment it is only available on FLIP-27 sources... does this include FlinkKafkaConsumer and FlinkKinesisConsumer?

It does not work for legacy or if applied after the source via DataStream#assignTimestampsAndWatermarks.

is another thing noted. does this mean that if i specify a watermarkstrategy via

assign_timestamps_and_watermarks

it will not let me use watermark alignment? does this mean i need a watermark strategy per source? since we are technically re-using the strategy and then passing it in e.g.

assign_timestamps_and_watermarks(watermark_strategy)

last question. is there any rule of thumb for how many seconds i should set my max_drift + watermark_interval? the docs specify 20 seconds and 1 second respectively. we have events that can be as late as 10-30 minutes. would this mean we are throttling sources for that long? thanks!

Both those consumers are not FLIP-27 so they don’t support it

damn ok gotcha. is KafkaSource FLIP-27?

Yep

sweet. thanks!

Hey @Felix Angell. I am also looking for a Flip-27 Kinesis Source to use in a HybridSource. Did you have any success here? It looks like this is still under development.

no success. luckily we have the same data set from kafka so we opted to use this instead 😬

ok, thank you!