suppose I created a realtime table with some kafka with ssl enabled configs .. now my kafka(Strimzi kafka) crashed or its ssl configs changed along with broker IP .. so can I point the same pinot table to the new kafka broker Ip and updated ssl configs without losing the old data .. Is backup & restore necessary while doing this or can directly update the table creation connfigs @Xiang Fu @Mayank cc: @Mohamed Sultan @Shailesh Jha @Mohamed Kashifuddin @Mohamed Hussain @Pugal

if Kafka topic partition offsets are persisted then you are fine

u mean I dont need to delete the table . .just to update the table configs where we give broekr url and ssl configs

As Xiang mentioned, if the offsets are consistent in old vs new Kafka then you are good. For example, let's say Pinot consumed until offset xx in old kafka. Does the new Kafka have the same offset for those messages? If so, yes. If not, then you need to redo the table

ok but as Xiang meantioned earlier that pinot stores offsets in zookeeper which comes with pinot helm installation .. so even if the kafka gets replaced .. the pinot setup zookeeper will be still there with the offsets . .so then we dont need to delete the table since nothing changed in the zookeeper of pinot

Pinot does do that. What I am saying is if old and new Kafka are not consistent (eg message m1 and offset o1 in old kafka is not the same as message m2 in offset o1 in new Kafka), then Pinot won't know about it. It will consume from the offset it saved in ZK, but that offset may not be correct for new Kafka

ok got it .. u mean the ordering of events in both kafka shuld exactly comply on a particular topic .. but since old kafka is lost.. and the new kafka will get only the latest data ..since old data is already consumed in pinot and the retention period of kafka was of only7 days.. so the same ordering of events cant be present on the new kafka .. it will have the latest data starting from the time it is created which is after old kafka crashed/its ssl config /broker url changed

I am also talking about offset. That number is not contiguous like 1, 2, 3, 4,... It is only monotonically increasing. So one can have 1, 3, 5... another one can have 2, 8, 9....

these few highlighted configs in the table c reation script got changed

Ok, let me ask you this. Are you doing this in production, or just playing around?

got ur point .. so in some way .. kafka is a single point of failure .. if kafka fails .. we have to take backup and restore only in new realtime table .. since i dont think its possible to create a new akfka instance having the same offsets of the same messages

If you have setup kafka with enough fault tolerance, then it should not fail

we are about to get into prod .. thus need to ensure that whatever we are planning is rock solid ..

👍