Is there a segment level query caching happening in pinot. If we fire the same metric query we see the result response time decrease from seconds to milliseconds. But the same resets to seconds are sometime/other queries. Is there a document where I can read more about it?

There is no caching, this is likely due to cold start