This message was deleted.

Druid does this automatically via checkpointing for seekable streaming sources

We have multiple consumers for kafka. Except Druid everything auto commits to Kafka. which helps us compare lag among consumers using metrics like "aws.kafka.sum_offset_lag". Druid doesn't auto commit to Kafka which makes it difficult to compare lag with other Consumers. Druid has "druid.ingest.kafka.lag" metric, which somehow not equivalent to "aws.kafka.sum_offset_lag" to compare lag with other Consumers. I'm using Datadog for monitoring purpose.

Druid doesn't use Kafka's builtin consumer group management; we manage our own offsets in Druid to ensure exactly-onceness