Hi everyone, I have a question regarding Flink sinks and fault tolerance.
My setup is:
• Source: Kafka
• Sinks:
• Kafka Sink
• S3 Sink
Both sinks consume from the same stream.
Goal:
I want both sinks to operate independently, meaning:
• If the S3 sink fails, the Kafka sink should continue writing.
• If the Kafka sink fails, the S3 sink should continue writing.
At the same time, I want to maintain exactly-once delivery guarantees.
My questions are:
Is it possible in Flink to configure multiple sinks so that they fail independently without causing the job to restart?
Since Flink checkpointing is global across the job graph, how would state management and checkpointing behave in this case?
Would appreciate any guidance
Thanks!