George Leonard
06/25/2025, 5:32 AMimport org.apache.flink.api.common.serialization.SerializableSerializer;
...
@Override
public SimpleVersionedSerializerSnmpSourceSplit getSplitSerializer() {
return new SerializableSerializer<>();
}
@Override
public SimpleVersionedSerializer<ListSnmpSourceSplit> getEnumeratorCheckpointSerializer() {
return new SerializableSerializer<>();
}
parent pom.xml
properties
flink.version1.20.1/flink.version
java.version17/java.version
maven-compiler.version3.11.0/maven-compiler.version
maven-shade.version3.5.2/maven-shade.version
maven-compiler.source${java.version}/maven-compiler.source
maven-compiler.target${java.version}/maven-compiler.target
project.build.sourceEncodingUTF-8/project.build.sourceEncoding
...
local pom.xml
dependency
groupIdorg.apache.flink/groupId
artifactIdflink-core/artifactId
version${flink.version}/version
scopeprovided/scope
/dependencyAndreas Bube
06/25/2025, 6:32 AMorg.apache.flink.api.common.SerializableSerializer which was added in Flink 2.0. Still. I don't think that's a useful class in this case as it doesn't implement the SimpleVersionedSerializer interface.
The code you posted above doesn't make much sense. You have to actually implement the serialization logic that de-/serializes your SnmpSourceSplit objects into byte arrays. For example:
public static class FooSerializer implements SimpleVersionedSerializerFoo {
@Override
public int getVersion() {
return 0;
}
@Override
public byte[] serialize(Foo foo) throws IOException {
try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream(baos)) {
out.writeBoolean(foo.isFoo());
out.flush();
return baos.toByteArray();
}
}
public Foo deserialize(int version, byte[] serialized) throws IOException {
try (ByteArrayInputStream bais = new ByteArrayInputStream(serialized);
DataInputStream in = new DataInputStream(bais)) {
boolean isFoo = in.readBoolean();
return new Foo(isFoo);
}
}
}
I find that current crop of LLMs are not very helpful with Flink implementations. They seem to lacking the proper training data, and often make quite fundamental mistakes.George Leonard
06/25/2025, 6:37 AMGeorge Leonard
06/25/2025, 6:37 AMAndreas Bube
06/25/2025, 6:40 AMGeorge Leonard
06/25/2025, 6:42 AMAndreas Bube
06/25/2025, 6:44 AMGeorge Leonard
06/30/2025, 5:51 AM