Hey, I want to use state to deduplicate items received from Kafka (topic itself might contain duplication). I already have experience with MapState, but in ListState there is no contains() method. Is there any way to use ListState efficiently to check if item is in state, without iterating a whole state?

I would recommend to just use the Flink Table deduplication query; that already contains your desired functionality. See https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/deduplication/

What if I'm not using Table API? Can I use this in regular KeyedProcessFunction?

No, you can't

@Martijn Visser is it not possible to have keyBy(uuid) -> FilterFunction with state (with ttl) for dedup ? uuid can be duplicated from source in kafka topic.

^ what Pankaj said, just use ValueState (can be a boolean) with a keyBy.

I was referring that you can’t use the Table deduplication operator in a KeyedProcessFunction. But you can definitely build your own implementation