Also, another question, maybe dumb as I am not a devOps exprt From the architecture docs, servers host segments, does this mean that the Persistent Volume of the server (in k8s) can hold hundreds of GB of data? In case of a realtime ingestion (kafka), If the server restarts before committing the data to the storage, will the consumer be aware of this and avoids committing the kafka offset? to ensure data consistency

Yes to both. Servers can host data as much as the disk allows (ymmv based on cpu/mem). Server restarts don’t cause any inconsistency