Hi, We are running a flink job in session mode and...
# troubleshooting
a
Hi, We are running a flink job in session mode and if one TaskManager pod within a Flink job fails, the overall Flink job is restarting. Is there a way that flink job can continue to run without interruption incase of any taskmanager pod failure ?
plusone 1
r
No. If a TaskManager running parts of a Flink job fails, the job needs to be restarted (if you have a job consisting of multiple disconnected DAGs, only the DAG affected by the failure will restart). A full restart is required to reset the state to the last successfully completed checkpoint
a
@rmetzger Thank you for the response πŸ™‚ One qq on RocksDB : We are using RocksDB as the state backend for our Flink application. Is it possible to extract data from RocksDB and save it as a CSV file? Requirement is to do data reconciliation between what we transmit to our Elastic Cluster and what Flink processes.
r
You can use the state processor api to inspect data in checkpoints and savepoints: https://nightlies.apache.org/flink/flink-docs-master/docs/libs/state_processor_api/
it is probably easier to have 2 sinks in your Flink job: one that writes to Elastic, one that writes into the CSV file
πŸ‘ 1
gratitude thank you 1