HCX - Network Extension and MON operations fail in the UI with JSONObject["networkId"] not found
search cancel

HCX - Network Extension and MON operations fail in the UI with JSONObject["networkId"] not found

book

Article ID: 448101

calendar_today

Updated On:

Products

VMware HCX

Issue/Introduction

In the HCX UI, any of the following operations fail immediately:

  • Enabling Mobility Optimized Networking (MON) on an extended network
  • Disabling MON on an extended network
  • Extending a network
  • Un-extending a network

The task reports the following error:

JSONObject["networkId"] not found

Data-plane symptom

Workloads on the affected segment can experience 100% packet loss or incorrect routing. While the HCX UI shows MON as disabled for the segment, the workload continues to forward traffic to the NSX VDR MAC address 02:50:56:56:44:52 instead of forwarding it over the Network Extension appliance to the source gateway.

Environment

  • VMware HCX 4.x
  • Azure VMware Solution (AVS) / Google Cloud VMware Engine (GCVE)
  • Network Extension segments with Mobility Optimized Networking (MON)
  • Destination site backed by NSX 4.x

Cause

The HCX database is out of sync with the NSX segment binding for the extended network. The HCX management plane no longer holds a valid record for the segment, so any operation that reads the network object fails immediately with JSONObject["networkId"] not found.

The ESXi data plane is not updated as part of the failed operation, so the workload port retains the stale Local Egress (LE) flag. The LE flag instructs the workload to send traffic to the NSX VDR, which is why traffic is misrouted even though HCX reports MON as disabled.

Resolution

Step 1 - Verify the data-plane state

  1. SSH to the ESXi host where the affected workload resides.
  2. Identify the port ID of the affected VM: net-stats -l
  3. Check the remoteRtr property on the DVS ports: net-dvs -l | grep "com.vmware.nsx.port.extraConfig.remoteRtr"
  4. Review the entry for the port belonging to the affected VM. An example of a port carrying the LE flag: com.vmware.nsx.port.extraConfig.remoteRtr = ###.###.###.### 02:50:56:48:43:58 02:50:56:56:44:52 00:00:00:00:00:00 LE , propType = POLICY

If the port reports LE while MON is shown as disabled in the HCX UI, a stale flag is present on the host and the HCX and NSX records for the segment are out of sync.

Step 2 - Rebuild the network extension

The MON toggle cannot be used to clear the condition, because the toggle itself fails with the JSONObject["networkId"] not found error. The extension must be removed and re-created.

  1. From the HCX Connector (source side), select Services > Network Extension.

    Note: Perform the unextend and re-extend from the HCX source/connector side. Performing this operation from the HCX destination/cloud side is not supported.

  2. Select the affected network and click Unextend Networks.
  3. If the standard unextend fails, click the ellipsis menu next to the Source Network entry and select Force Unextend Networks. Confirm the action in the dialog box.
  4. If Force Unextend was used, verify that the L2E segment was removed from the destination NSX. If it persists:
    • Log in to the NSX UI as the admin user.
    • Navigate to the Networking tab and select Segments.
    • Locate the L2E segment and confirm that no VMs are attached to it.
    • Click the three vertical dots next to the segment and select Delete.
  5. Re-extend the network from the HCX Connector, and re-enable MON if it is required.

Step 3 - Confirm the flag matches the intended state

Re-run the check from Step 1 on the ESXi host:

net-dvs -l | grep "com.vmware.nsx.port.extraConfig.remoteRtr"

Confirm that the remoteRtr property on the workload port matches the intended MON configuration. The LE flag is expected only when MON is enabled on the segment.

Additional Information

Related KB - Network traffic for a VM with HCX MON (Mobility Optimized Networking) disabled is being forwarded to the NSX VDR gateway causing network issues.