Your workflow is , lead comes in -> you call the lead (outbound ) , the lead could call back this number no one else can since the phone number is not public only the lead can see it ,if the lead call back you can check if the number is on the CRM or not before the call starts if not you can reject the call ,
You can remove the phone number and buy a new one , the new charges will stop after you remove it.