Hello, for kafka ingestion, are replicas necessary to ensure that I have exactly once ingestion even when a task fails? Or is it more for high availability to queries?

The replica that you use in TASKS are for task replication, for replica:2 2 tasks will be running, but only one task for each partition actually publishes the segments. Segment replication is for queries.

To add to what Bibek says -- Druid uses the "consumer assignment" model for pulling data from the stream, i.e. Druid keeps track of the Kafka partition (or Kinesis shard) offsets it has read from, these offsets are stored in the metadata DB where the published segment data is also kept. Druid moves the highwatermark offset when it publishes a segment ... this is done in an atomic RDBMS transaction in the metadata DB. If an ingestion task were to fail before it publishes a segment to deep storage, then the current information in the metadata DB reflects the last segment to be published, and the offsets that correspond to the data up to that last segment. The replacement ingestion task will then pick up from that point and start reingesting that data again. So, even with a single ingestion task running, the way Druid keeps track of its work in the metadata DB ensures that each piece of event data will be ingested from the stream and into Druid exactly once.

🤠