Hi all, the data pipeline I am experimenting with ...
# troubleshooting
s
Hi all, the data pipeline I am experimenting with is
MySQl --> Kafka --> FLink --> Application database
, one of the tables in source MySQL database has datatype
decimal(14,2)
, when this column data is transmitted to Kafka (ex - 500.00), I can see the values are getting stored correctly (500.00), however when this column is being read from Flink, it multiplies it with 100 for some reason (it becomes 50000.000000000000000000, also, not sure why 18 decimal points) , am I doing the deserialization wrong somehow?
Copy code
MysQL Data Type: decimal(14,2)
Kafka Confluent Avro Format Data Type: 
"type":[
    "null",
    {
        "type": "bytes",
        "logicalType": "decimal",
        "precision": 14,
        "scale": 2,
        "connect.name": "org.apache.kafka.connect.data.Decimal",
        "connect.parameters":
        {
            "connect.decimal.precision": "14",
            "scale": "2"
        },
        "connect.version": 1
    }
]
Flink SQL Data Type: decimal(14,2)
Flink version: 1.15.4
Kafka Version: 2.8.1
Flink connector: kafka
Flink connector 'value.format' = 'debezium-avro-confluent'


Example column value
--------------------
MySQL: 500.00
Kafka: 500.00
Flink: 50000.000000000000000000
This seems to be fixed by providing correct precision and scale at source table