Hi, We are using flink job (version 1.16) to consu...
# troubleshooting
n
Hi, We are using flink job (version 1.16) to consume message from kafka and push it back to kafka after processing. We are encountering below error in the jobmanager and flink job restarts after that. Is this error related flink or any configuration to recover from this error ERROR akka.remote.Remoting [] - Association to [akka.tcp://flink@10.101.212.188:6122] with UID [590651498] irrecoverably failed. Quarantining address. java.util.concurrent.TimeoutException: Remote system has been silent for too long. (more than 2 days) at akka.remote.ReliableDeliverySupervisor$$anonfun$idle$1.applyOrElse(Endpoint.scala:454) ~[flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.remote.ReliableDeliverySupervisor.aroundReceive(Endpoint.scala:249) ~[flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_75517d49-aee2-4205-be5b-dc2b88de8069.jar:1.16.0] at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]