Ricky Saltzer
08/23/2023, 9:43 PMCaused by: java.lang.IllegalArgumentException: Key group 75 is not in KeyGroupRange{startKeyGroup=0, endKeyGroup=63}. Unless you're directly using low level state access APIs, this is most likely caused by non-deterministic shuffle key (hashCode and equals implementation).
The only KeySelector I'm using is returning the following, which is definitely deterministic. The AggregationType is a valueless enum
data class MetricKey(
val name: String,
val aggregationType: AggregationType,
val timestamp: Long,
)
The event stream is being fed into my KeyedProcessFunction
class AggregationProcessor : KeyedProcessFunction<MetricKey, Metric, AggregationResult>()
This class does hold several AggregatingStateDescriptors which relies on the key so that aggregations are performed with the data grouped by the key.
Any idea what to look for?Ricky Saltzer
08/24/2023, 2:47 PMAggregationType `Enum`causing weird behavior when serializing / hashcodes. I instead just stored the ordinal and now its fine