VMware NSX for vSphere 6.x Edge loses connectivity after upgrade with the error: The requested object could not be found

Products

• VMware NSX

Issue/Introduction

Symptoms:

• Upgrading VMware NSX for vSphere 6.x Edge fails
• You see the error similar to:
  
  The requested object: 5000-Transit_Internal could not be found, Object identifiers are case sensitive

• In the NSX Manager Log Bundle, you see entries similar to:
  
  Edge Gateway', Operation:'UPGRADE', Resource:'dlr01', Time:'Sat Apr 25 16:58:28.663 GMT 2015'
  2015-04-25 16:58:28.666 GMT WARN http-nio-127.0.0.1-7441-exec-60 RemoteInvocationTraceInterceptor:87 - Processing of VsmHttpInvokerServiceExporter remote call resulted in fatal exception: com.vmware.vshield.edge.facade.EdgeFacade.upgradeEdge
  com.vmware.vshield.vsm.exceptions.ObjectNotFoundException:
  core-services:202:The requested object : virtualwire-1 could not be found. Object identifiers are case sensitive.
  at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl.resolveVirtualWireToPgId(VirtualWireServiceImpl.java:1420)
  at com.vmware.vshield.edge.publish.VcOperationsUtils.getEdgeVnicDvpg(VcOperationsUtils.java:2377)
  at com.vmware.vshield.edge.service.EdgeConfigServiceImpl.validateVnicPortSpecificProperties(EdgeConfigServiceImpl.java:519)
  at com.vmware.vshield.edge.service.EdgeConfigServiceImpl.validateVnics(EdgeConfigServiceImpl.java:590)
  at com.vmware.vshield.edge.service.EdgeConfigServiceImpl.prepareConfig(EdgeConfigServiceImpl.java:264)
  at com.vmware.vshield.edge.service.EdgeServiceImpl.populateAllFeatureConfigs(EdgeServiceImpl.java:547)
  at com.vmware.vshield.edge.service.EdgeServiceImpl.doUpgrade(EdgeServiceImpl.java:882)
  at com.vmware.vshield.edge.service.EdgeServiceImpl.upgradeEdge_aroundBody44(EdgeServiceImpl.java:917)
  at com.vmware.vshield.edge.service.EdgeServiceImpl$AjcClosure45.run(EdgeServiceImpl.java:1)
  at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
  at com.vmware.vshield.vsm.aspects.security.VsmSecuredAspect.secureCheck(VsmSecuredAspect.java:105)
  at com.vmware.vshield.vsm.aspects.security.VsmSecuredAspect.ajc$inlineAccessMethod$com_vmware_vshield_vsm_aspects_security_VsmSecuredAspect$com_vmware_vshield_vsm_aspects_security_VsmSecuredAspect$secureCheck(VsmSecuredAspect.java:1)
  
  2015-04-25 17:07:42.344 GMT ERROR TaskFrameworkExecutor-19 VirtualWireServiceImpl:1417 - No valid backing for virtual wire virtualwire-1 found in RP resgroup-82 in cluster domain-c81 on switch dvs-127.
  2015-04-25 17:07:42.345 GMT ERROR TaskFrameworkExecutor-19 PublishTask:347 - Failed jobId 'jobdata-76607' for edge 'edge-X' during publishing.
  com.vmware.vshield.vsm.exceptions.ObjectNotFoundException:
  core-services:202:The requested object : virtualwire-1 could not be found.
  Object identifiers are case sensitive.
  at com.vmware.vshield.vsm.vdn.service.VirtualWireServiceImpl.resolveVirtualWireToPgId(VirtualWireServiceImpl.java:1420)
  at com.vmware.vshield.edge.publish.VcOperationsUtils.getEdgeVnicDvpg(VcOperationsUtils.java:2377)
  at com.vmware.vshield.edge.utils.SubInterfaceUtils.populateOldNewTrunkVnicInfo(SubInterfaceUtils.java:426)
  
  Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX for vSphere 6.1.x
VMware NSX for vSphere 6.0.x

Cause

This issue occurs due to missing portgroup entries in the virtualwire table for one of the vDS (Edge Cluster) in the setup.

Resolution

To resolve this issue, follow these steps:

1. Recreate the missing port group entries on the vSphere Distributed Switch (vDS).
2. Find the scope to repair using REST API.
   
   GET on <NSX_Manager_IP>/api/2.0/vdn/scopes
3. Run a repair API on the vdsScope to recreate the missing port groups.
   
   POST on http://NSX_Manager_IP/api/2.0/vdn/scopes/<id>?action=repair

Additional Information

Collecting diagnostic information for VMware NSX for vSphere 6.x

Impact/Risks:
Creating a new NSX edge devices or upgrading NSX edge devices fails