Hi everyone, I am working on a use case that invol...
# troubleshooting
y
Hi everyone, I am working on a use case that involves writing avro records to a kafka topic, from pipeline-a using DataStream API (Without schema registry, performance concerns). Then, using pipeline-b i create a table with the expected format and try to consume data from the topic to go down the stream. I have an issue with serialization, as the avro connector for Table API generates a schema with name of record = "record" and namespace="org.apache.flink.avro.generated". Have anyone experienced similar scenario, and can maybe advise what are my options here?
p
By the docs, defining an Avro schema is not supported yet. What are the performance concerns for the schema registry? Perhaps it's okay to handle it adding a caching layer.
If you need to follow this path, you could source the data on pipeline-b using datastream api and then converting it to table api. But I think this will be a bigger performance detractor than having the schema registry.