Hi, is there any approach to view the contents st...
# general
k
Hi, is there any approach to view the contents stored on segment ?
m
Other than some metadata, the segments are in binary format. What specifically are you looking for ?
k
I was exploring on how data is sharded to form segments. Is there any information that can help me with this ?
m
If you do a batch ingestion you'll end up with one segment per file that you're ingesting from. If you're doing a real-time ingestion, you'll end up with segment files per partition of the stream that you're ingesting from. Those segments will be flushed based on number of messages/time/size. (https://docs.pinot.apache.org/operators/operating-pinot/tuning/realtime#tuning-realtime-performance)
👍 1