Hi :wave: How much storage is on average required ...
# generals
Sahar
01/21/2022, 3:25 PMHi 👋 How much storage is on average required for Pinot relative to source data size? For instance if my source database is 1TB, how much EBS do I need on AWS for the Pinot cluster? I'm imagining it'd be more than the source size due to replication and indexing?
k
Kishore G
01/21/2022, 3:27 PM
whats the source format?
s
Sahar
01/21/2022, 3:31 PMMysql database
Sahar
01/21/2022, 3:31 PMMysql binlogs get consumed by Debezium and get published as Avro files to Kafka topics which are then ingested by Pinot
k
Kishore G
01/21/2022, 4:13 PM
so avro files you will see 3x compression
Kishore G
01/21/2022, 4:14 PM
so plan for originalsize/3 *replication factor/num-servers for each ebs volume
s
Sahar
01/21/2022, 4:17 PMok makes sense, thank yoiu