(Pyflink) Is it possible to use two different Json...
# troubleshooting
a
(Pyflink) Is it possible to use two different JsonRowSerializationSchemas for key/value when using the Kafka connector in DataStream? For instance, say I am using
Copy code
serialization_schema = JsonRowSerializationSchema.builder().with_type_info(
    type_info=Types.ROW_NAMED(["app", "bytes", "cid"], [Types.STRING(), <http://Types.INT|Types.INT>(), Types.STRING()])).build()
For the value, but I don't want "bytes" in my key because it is highly cardinal so I use:
Copy code
key_serialization_schema = JsonRowSerializationSchema.builder().with_type_info(
    type_info=Types.ROW_NAMED(["app", "cid"], [Types.STRING(), Types.STRING()])).build()
But the connector dislikes this as it says it cannot convert the bytes field to string. Is there a way to ignore this bytes field and only select the fields I want?
j
I'm not sure of your setting but I'm able to have different type info for the key and value.
a
When I select the field like that, it literally just chooses the first field in the row rather than actually picking the field with that name
j
I find it is not easy to manipulate row, I converted into a data class instance.
a
I think that makes sense, maybe I will approach it that way. Thanks
Yeah, I manipulated the row using a data class and this still doesn't work for me πŸ˜„ It's bizarre. It wants to select each field from the row rather than the subset I have identified. Guess I can't use Kafka keys.