While open source pinot is still in minor release version, LinkedIn has deployed pinot on multiple site-facing use cases, serving 100s of millions of members. Breaking backward compatibility can be bad, and should not be done unless absolutely unavoidable. Also, we should make sure that any new rollouts can be backed out safely (consider the steps: Roll out new code, upgrade config to use new ports, oops ne code found some other bug, so roll it back but still with updated config). So, your proposal of deprecating the old config for 0.5.0 and removing it in 0.6.0 seems to be the way to go. Extra configs should be ignored.