I have a real time table ingesting from kafka. Does the offsets in a segment metadata correspond to the kafka stream offsets? Or does pinot have its own internal offsets?

Yes, offsets recorded in segment metadata are those of the stream (but represented in a string form). Since you are using Kafka, the string should just be the same number (except in string form) in the metadata.

Hey Subbu, Thanks for the swift reply. Appreciate it. So , we can update a table in place to go back to a particular offset right? Change the

stream.kafka.consumer.prop.auto.offset.reset

to a particular timestamp. However, is there a way to change to a particular offset instead of timestamp?

No way to change it to a particular offset (that will have to be on a per-partition basis).

Got it. Thanks

It is up to the stream implementation to provide a timestamp based reset. I believe most Kafka implementations do provide it, and it is an approximation (so, it will rewind the stream enough to include the time you specify).