Marco Villalobos
08/06/2025, 4:15 AMfinal JsonDeserializationSchema<ObjectNode> jsonDeserializationSchema = new JsonDeserializationSchema<>(ObjectNode.class);
final KafkaSource<ObjectNode> kafkaSource = KafkaSource.<ObjectNode>builder()
.setBootstrapServers(kafkaBootstrapServers)
.setTopics(kafkaTopic)
.setGroupId(kafkaGroupId)
.setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST))
.setValueOnlyDeserializer(jsonDeserializationSchema)
.build();
but when I run it I get the following stack trace:
2025-08-06 04:09:44,311 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: flame-inf kafka source -> Sink: Print to Std. Out (1/2)#0 (e119be7364394210a5290c812216a46b_9a1186299580bdec0729e3077675ce1a_0_0) switched from RUNNING to FAILED with failure cause:
java.io.IOException: Failed to deserialize consumer record due to
at org.apache.flink.connector.kafka.source.reader.KafkaRecordEmitter.emitRecord(KafkaRecordEmitter.java:56) ~[blob_p-da5c3d686c1802bf53f64aa7321d844d6ada3eb1-fc90b5255bf6e1738c8dc5257b589a2f:?]
at org.apache.flink.connector.kafka.source.reader.KafkaRecordEmitter.emitRecord(KafkaRecordEmitter.java:33) ~[blob_p-da5c3d686c1802bf53f64aa7321d844d6ada3eb1-fc90b5255bf6e1738c8dc5257b589a2f:?]
at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:203) ~[flink-connector-files-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:443) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:638) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:973) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:917) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:949) [flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) [flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) [flink-dist-1.20.2.jar:1.20.2]
at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Caused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Conflicting setter definitions for property "all": com.fasterxml.jackson.databind.node.ObjectNode#setAll(com.fasterxml.jackson.databind.node.ObjectNode) vs com.fasterxml.jackson.databind.node.ObjectNode#setAll(java.util.Map)
at [Source: (byte[])"{"type": "xxxx", "sha256": "not used", "frame": 1, "top_left": {"x": 101, "y": 1}, "bottom_right": {"x": 390, "y": 189}}"; line: 1, column: 1]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1909) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:268) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:648) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4861) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4731) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3738) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.formats.json.JsonDeserializationSchema.deserialize(JsonDeserializationSchema.java:69) ~[flink-json-1.20.2.jar:1.20.2]
at org.apache.flink.api.common.serialization.DeserializationSchema.deserialize(DeserializationSchema.java:82) ~[flink-dist-1.20.2.jar:1.20.2]
at org.apache.flink.connector.kafka.source.reader.deserializer.KafkaValueOnlyDeserializationSchemaWrapper.deserialize(KafkaValueOnlyDeserializationSchemaWrapper.java:51) ~[blob_p-da5c3d686c1802bf53f64aa7321d844d6ada3eb1-fc90b5255bf6e1738c8dc5257b589a2f:?]
at org.apache.flink.connector.kafka.source.reader.KafkaRecordEmitter.emitRecord(KafkaRecordEmitter.java:53) ~[blob_p-da5c3d686c1802bf53f64aa7321d844d6ada3eb1-fc90b5255bf6e1738c8dc5257b589a2f:?]
... 14 more
I'm also wondering if it is because print does not know how to serialize ObjectNode? Must I register a serializer for it or something?
I even mapped the source into a string before the print like this:
`
jsonNodeSource.map(BaseJsonNode::toString).print();
but I still get this error:
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Conflicting setter definitions for property "all": com.fasterxml.jackson.databind.node.ObjectNode#setAll(java.util.Map) vs com.fasterxml.jackson.databind.node.ObjectNode#setAll(com.fasterxml.jackson.databind.node.ObjectNode)