Hey! Im trying to migrate to KafkaSink but seing a...
# troubleshooting
z
Hey! Im trying to migrate to KafkaSink but seing a massive increase in Heap Memory being taken due to the generation of new TransactionalIds for every checkpoint. My jobs checkpoint every second. I cannot compromise on the number of subtasks, the checkpoint interval maybe. I saw a similar issue here, but a resolution was not reached there. How dis everyone else dealing with this increase in memory because of kafkaSInk as opposed to FlinkKafkaProducer? My Kafka version is 3.4 and Flink version is 1.14.3