When deleting a port group you may observe below symptoms
The concerned port group has reference to one of the VM snapshots which was taken earlier when that port group was in use and prevents from deleting that port group now with no VMs using that port group.
To resolve this issue, if the snapshot is no longer required, remove it from within Snapshot manager.
Also, follow this reference KB for more information on the issue resolution Summary tab shows multiple network port groups that are not used by the virtual machine.