Ola Theander
10/03/2023, 10:02 AMjava.time.Instant
when implementing the stream process in Scala (doing the equivalent in Java works fine).
I have a class like:
case class Person(name: String, age: Int, created: java.time.Instant)
and given the implementation added in https://issues.apache.org/jira/browse/FLINK-10127 I expected that the class would serialize just fine (again in pure Java it works) but I get:
java.io.UncheckedIOException: java.io.IOException: Serializing the source elements failed: java.lang.IllegalArgumentException: Unable to create serializer "com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.time.Instant
Serialization trace:
created (flink.Person)
at org.apache.flink.streaming.api.functions.source.FromElementsFunction.setOutputType(FromElementsFunction.java:158)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySetOutputType(StreamingFunctionUtils.java:84)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.setOutputType(StreamingFunctionUtils.java:60)
...
..
Caused by: java.io.IOException: Serializing the source elements failed: java.lang.IllegalArgumentException: Unable to create serializer "com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.time.Instant
Serialization trace:
created (flink.Person)
at org.apache.flink.streaming.api.functions.source.FromElementsFunction.serializeElements(FromElementsFunction.java:135)
at org.apache.flink.streaming.api.functions.source.FromElementsFunction.setOutputType(FromElementsFunction.java:156)
... 23 more
Caused by: com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Unable to create serializer "com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.time.Instant
Serialization trace:
created (flink.Person)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599)
at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:362)
at org.apache.flink.streaming.api.functions.source.FromElementsFunction.serializeElements(FromElementsFunction.java:132)
... 24 more
Caused by: java.lang.IllegalArgumentException: Unable to create serializer "com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.time.Instant
at com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:48)
at com.esotericsoftware.kryo.factories.ReflectionSerializerFactory.makeSerializer(ReflectionSerializerFactory.java:26)
... 28 more
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
... 35 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final long java.time.Instant.seconds accessible: module java.base does not "opens java.time" to unnamed module @6cdfec5c
at java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:391)
at java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:367)
...
..
If I remove the created
attribute so that I just have case class Person(name: String, age: Int)
or if I implement a non-idiomatic POJO-style Scala class:
class Person {
private var _name: String = _
private var _age: Int = _
private var _created: java.time.Instant = _
def this(name: String, age: Int, created: java.time.Instant) = {
this()
_name = name
_age = age
_created = created
}
def name: String = _name
def age: Int = _age
def created: java.time.Instant = _created
def name_=(name: String): Unit = _name = name
def age_=(age: Int): Unit = _age = age
def created_=(created: java.time.Instant): Unit = _created = created
}
it works fine.
What's also surprising is that PojoTestUtils.assertSerializedAsPojo[Person]
as well as PojoTestUtils.assertSerializedAsPojoWithoutKryo[Person]
pass but still when actually serializing it seems to fall back to Kryo which then with the above exception.
Is this perhaps a bug even if I realize that Scala is not mainstream? I.e. considering that Scala case-classes are supported and so are java.time.Instant
.