Is it possible to delete a Cost plan with Plan of Record (POR)?
Yes, from CA PPM 14.3 Version its possible to delete Cost plan with Plan of Record (POR) and allowing you to have zero cost plans within the project.
This is a new feature mentioned in 'Change Impact and Upgrade' Guide, unfortunately for older versions is not possible to delete it.
For older versions, create a new cost plan within the same project, mark the new one as POR and then you can delete the first cost plan because it is not a POR. You will still be left with at least 1 cost plan marked as a POR in the older versions.