@Somanshu Jindal For prod, here is a good setup

controller 
- min 2 (for fault tolerance) ideal 3 
- 4 core, 4 gb (disk space should be sufficient for logs and temp segments) - 100 GB
Broker
- Min 2, add more nodes as needed as later to scale
 - 4 core, 4gb (disk space should be sufficient for logs) - 10GB min
Zookeeper (cluster mode), 
- min 3 (this is where the entire cluster state is stored)
- 4 gb, 4 core,  disk space sufficient to store logs, transaction logs and snapshots. If you can afford, go with ssd if not disk will be fine. 100GB

Pinot server
- Min 2 (this is where the segments will be stored), you can add more servers anytime without downtime
- 8 core, 16 gb, SSD boxes (pick any size that works for your use case (500 gb to 2TB or even more). 
- If you are running on cloud, you can use mounted ssd instead of local ssd

Hey Kishore, If i am using S3PinotFS would I still need ssd boxes with my pinot server ?

yes, it can be local or remote mounted, like EBS