NSX opaque networks missing from vCenter inventory after vCenter certificate rotation
search cancel

NSX opaque networks missing from vCenter inventory after vCenter certificate rotation

book

Article ID: 434473

calendar_today

Updated On:

Products

VMware NSX VMware Aria Operations (formerly vRealize Operations) 8.x

Issue/Introduction

After a vCenter certificate rotation, NSX segments no longer appear as opaque networks in the vCenter inventory. Virtual machines (VMs) that were previously connected to NSX-backed opaque networks show the network as unavailable or display null references for their network backing.

The following symptoms are observed:

In the vpxd logs, every reference to the opaque network objects returns a ManagedObjectNotFound error:

error vpxd[######] [Originator@6876 sub=Authorize opID=########-####-####-####-############]
MoRef: vim.OpaqueNetwork:network-o#### not found.
Error: N5Vmomi5Fault21ManagedObjectNotFound9ExceptionE
(Fault cause: vmodl.fault.ManagedObjectNotFound)

These errors appear continuously and persist across log rotations, indicating the opaque networks are not registered in the vCenter inventory.

Additionally, vpxd logs show NSX port group creation attempts being rejected with validation failures:

warning vpxd[######] [Originator@6876 sub=MoDVSwitch opID=########]
Validation check failed for create NSX port group:
(vim.dvs.DistributedVirtualPortgroup.ConfigSpec) {

This indicates NSX is attempting to register segments as port groups through the DVSwitch subsystem, but vCenter is not accepting the requests.

Steps to validate

  1. Check the vpxd logs for ManagedObjectNotFound errors referencing OpaqueNetwork objects:
grep "OpaqueNetwork.*not found\|ManagedObjectNotFound.*network-o" /var/log/vmware/vpxd/vpxd*.log
  1. Check for valid (non-error) opaque network references. If the only references are ManagedObjectNotFound errors, the opaque networks are not in the vCenter inventory:
grep "OpaqueNetwork" /var/log/vmware/vpxd/vpxd*.log | grep -v "not found\|NotFound\|ManagedObjectNotFound"
  1. Check for NSX port group validation failures:
grep "Validation check failed for create NSX port group" /var/log/vmware/vpxd/vpxd*.log
  1. On the NSX Manager, check the CmInventoryService logs for the compute manager connection status. Look for cmConnectionStatus=DOWN entries:
grep "cmConnectionStatus" /var/log/proton/*.log
  1. Verify the NSX compute manager connectivity is active by checking for successful NSX extension (com.vmware.nsx.management.nsxt) EAM logins in the vCenter journalctl or EAM logs. The presence of successful logins indicates the NSX-to-vCenter control plane is functioning.

Environment

  • VMware NSX 4.x
  • VMware vCenter Server 8.x

Cause

A vCenter certificate rotation breaks the SSL trust between NSX and vCenter, causing the NSX compute manager connection to transition to a DOWN state (cmConnectionStatus=DOWN). This is visible in the NSX Manager CmInventoryService logs.

After the certificate trust is re-established and the compute manager connection recovers to an UP state, the opaque network re-synchronization may not complete successfully. The NSX cm-inventory plugin attempts to re-register segments as port groups in vCenter, but vCenter rejects the creation requests with "Validation check failed for create NSX port group" errors at the MoDVSwitch validation layer.

This results in opaque networks remaining absent from the vCenter inventory despite the NSX compute manager being connected and actively attempting registration.

Resolution

If the symptoms above match the environment, this appears to be a vCenter-side issue rather than an NSX issue. Open a support request with the SDDC or Compute team and provide the following:

  • vCenter support bundle (vm-support)
  • NSX Manager support bundle
  • Output of the vpxd log analysis showing the ManagedObjectNotFound errors and validation failures
  • The CmInventoryService log entries showing the cmConnectionStatus=DOWN and UP transition history
  • Timeline of the certificate rotation event
  • Any related SR numbers for the certificate rotation or compute manager connectivity issue

Workaround

While the underlying opaque network registration remains in a failed state, adding the NSX Manager as a cloud account in VMware Aria Automation allows the NSX segments to be visible for automation and provisioning purposes. This does not resolve the opaque network sync in vCenter but provides an alternative path for segment visibility.

To configure this:

  1. In VMware Aria Automation, navigate to Infrastructure > Connections > Cloud Accounts.
  2. Click Add Cloud Account and select NSX-T as the account type.
  3. Enter the NSX Manager FQDN or IP, credentials, and associate it with the existing vCenter cloud account.
  4. After the cloud account syncs, NSX segments become available as network resources in Automation Assembler and can be used in cloud templates and provisioning workflows.

This workaround does not restore opaque network references for VMs in vCenter inventory. VM network backing still shows null opaque network references until the underlying sync issue is resolved.

If the error persists after following these steps, contact Broadcom Support for further assistance.

Additional Information

  • After the compute manager connection recovers, SSL handshake failures (sslv3 alert bad certificate) may still appear in the NSX Manager logs, which can indicate incomplete certificate trust restoration.
  • The cmPluginStatusData=null entries in CmInventoryService logs indicate the cm-inventory plugin is not running on that NSX Manager node during those intervals, which prevents inventory sync operations from being initiated.
  • The NSX compute manager UUID can be identified in the CmInventoryService log entries and cross-referenced with the NSX Manager API (GET /api/v1/fabric/compute-managers) to confirm the compute manager configuration.
Powered by Wolken Software