To resolve this issue, disconnect the port and then retry the operation.
Note: Usually, the vSphere Client detects this issue and does not issue the task to vCenter Server. But, if you use API to perform these tasks, the script may not perform these checks before issuing the task to vCenter Server and, therefore, the issue is detected by vCenter Server. If you are using API scripts, ensure that the port, portgroup, or switch is free before attempting to delete it.