brokerDeleteInactiveTopicsEnabled=true do i need to make it as false ?

yes, if you don't want this to happen.

Can we specifically set it for Some Topics ?

yes. check

pulsar-admin namespaces set-inactive-topic-policies

(reference) and

pulsar-admin topics set-inactive-topic-policies

(reference)

Sure. this Worked