NSX 4.x
During segment deletion, intermittent issues (e.g. connectivity issue) may cause logical segment to get to failed state.
Logical switch auto rectification mechanism (NSX Manager's internal process) does not work for logical switches in failed state, hence auto rectification mechanism may fail to automatically rectify logical switches which are in FAILED state.
Due to this, port groups will not get cleared from the vCenter.
This issue has been resolved in NSX 4.1.2.5, and NSX 4.2.0 and higher.
Please refer to VMware NSX 4.1.2.5 Release Notes, "Fixed Issue 3413790" discussed in section "Resolved Issues".
Workaround:
To workaround this issue, stale/orphan port groups must be manually deleted from the vCenter's database. For assistance with this task, please open a new case with Global Support team with reference to this KB article.