Hello Folks,
We are frequently seeing following er...
# troubleshootings
Sumit Nekar
03/11/2023, 5:16 AMHello Folks,
We are frequently seeing following error when the FlinkKafkaProducer is handling high throughput. We are using flink 1.13.6. Any suggestions on tuning the flink kafka producer configs or how should I configure producer to retry the failed request?
Copy code
Caused by: org.apache.flink.streaming.connectors.kafka.FlinkKafkaException: Failed to send data to Kafka: The server disconnected before a response was received.
Caused by: org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received.m
Martijn Visser
03/13/2023, 8:40 AMI would recommend upgrading to a more recent version of Flink; FlinkKafkaProducer has also been deprecated in more recent Flink versions in favour of KafkaSInk
s
Sumit Nekar
03/13/2023, 10:58 AMSure. Thanks
Sumit Nekar
03/20/2023, 5:25 AMAre there any improvements in the new KafkaSource and KafkaSink connectors in the latest versions and can they help to avoid such issues?
m
Martijn Visser
03/20/2023, 12:17 PMThey use the Source- and Sink APIs of Flink, which are developed for a unified batch and streaming effort. They also have had more recent updates of the Kafka Clients, compared to the older versions
s
Sumit Nekar
03/20/2023, 12:26 PMGreat. Thanks
4 Views