In large NSX environments, the first incoming vMotion of an NSX overlay connected VM causes network downtime for an extended period.

Products

• VMware NSX

Issue/Introduction

• NSX environment has a large number of segments (possibly 1000 or more) connecting to the same T1/T0 DR
• Hosts are prepared for NSX but have no NSX Overlay connected VMs or may have just exited maintenance mode
• vMotion of the first NSX Overlay connected VM has a significant network outage during the process
• Subsequent vMotions of NSX Overlay VMs to the host have little to no downtime.

Environment

VMware NSX

Cause

Due to the design of NSX, this behavior is expected. In large environments (for example, over 1000 segments connecting to the same T1/T0 DR) where no T1 SR is required there is a delay in initializing all segments and related network configurations.

Resolution

This is a condition that may occur in a VMware NSX environment.

 

Workaround: 

Create a segment port that is unused and applied to the host. This will maintain the nsx configuration on the host while in maintenance mode or if no NSX overlay VMs are on the host. 

• On the host, run the following command on the CLI terminal (as root through ssh) 

nsxcli -c get node-uuid

 

• Take note of the output (node uuid). 
• Within NSX UI, from the "Networking" tab, select "Segments". 
• You can create a segment specifically for this or use an existing segment. From the segment that will be used, click the three ellipses for the segment and click "Edit". 
• Under the "Ports / Interfaces" column, click "Set" (if no ports exist) or click on the number (which is the number of ports already on the segment). 
• Under the "Segment Ports" click "Add Segment Port" 
   - Provide a Port Name 
   - Under the "Connected To" column, apply the host node uuid (obtained from the host "nsxcli -c get node-uuid" command) 
     - ID : <node-uuid>
     - Type: Static
     - Context ID: <node-uuid> 
•  Click "Save" and then click "Close"
•  Click "Close Editing" on the segment window. 

 
This can also be achieved via API call 

 
 PUT https://{{url}}/policy/api/v1/infra/segments/<segment-ID>/ports/<Port-Name>

{
  "attachment": {
    "id": "<node-uuid>",
    "type": "STATIC",
    "context_id": "<node-uuid>"
  },
  "admin_state": "UP",
  "resource_type": "SegmentPort",
  "id": "<Port-Name>",
  "display_name": "<Port-Name>",
  "path": "/infra/segments/<segment-ID>/ports/<Port-Name>",
  "relative_path": "<segment-ID>",
  "parent_path": "/infra/segments/<segment-ID>"
}