Information on NSX for vSphere, EAM and TEPs

Products

• VMware NSX for vSphere

Issue/Introduction

This solution provides information on EAM, Impact on NSX when doing normal vCenter Server operations and VTEPs/VXLAN.

 

Environment

VMware NSX for vSphere 6.4.x

Resolution

What is EAM?

NSX Manager leverages EAM to prepare a given NSX Cluster by creating an Agency with Scope that covers that Cluster. EAM is responsible for most of the cluster level operations within NSX. Installing vibs and creating Agents (GI vms, PA VMs, etc.) are examples of what EAM does to a Cluster. NSX is hard coded to use port 8089 to talk with EAM, only default ports can be used with an EAM installation.

 

What is an Agency?

AN AGENCY IS A CLUSTER DEFINITION WHICH CAN BE SEEN IN WEBCLIENT OR EAM MOB

To list the agencies for each cluster go to: vSphere Webclient > Administration > vCenter Server Extensions >vSphere ESX Agent Manager >Manage To list the agencies in the mob: https://<vCenter_IP>/eam/mob/ EAM is just tables in the vCenter Database Table name: vpx_ext_data

Deleting or destroying Agencies Destroying NSX agencies in ESX agent manager causes the NSX vibs to get uninstalled from the corresponding vSphere Cluster To destroy an Agency un-prepare a cluster from within NSX (vSphere Webclient > Networking and Security > installation > Host Preparation > Uninstall) Agencies can be removed directly from the webclient under the ESX Agent Manager page Agencies can be removed from the EAM Mob (https://<vCenter_IP>/eam/mob/ > Click on an Agency > Destroy Agency) Removing any agency will cause the entire cluster to be uninstalled.

 

Impacting NSX by doing normal vCenter Server operations

Operation	Impacts	Is it safe
Restarting Management services (services.sh restart)	This will restart netcpa and vsfwd which will re-connect the rabbitmq service with NSX manager, and re-connect netcpa with the controllers.	YES
Reinstalling vCenter with the Same DB	On earlier versions e.g. VC 6.0 reinstalling vCenter against the same DB caused the vCenter UUID to change. EAM is not resilient to vCenter UUID changes and hence it is possible that NSX cannot prepare any new ESXi hosts. this issue is resolved on later versions of vCenter.	Most of the time

 

What are VTEPs/VXLAN

VXLAN is the communication Channel NSX creates between ESXi hosts, this channel is provided by VTEPs created on each ESXi host.

VTEP interfaces When a cluster is prepared, a vmkernel interface (VTEP) is added to every host in the cluster. VXLAN encapsulated packets are sent out on this interface, changes to this interface i.e. IP settings could cause network disruptions, these vmks should never be changed.

 

VXLAN IS THE COMMUNICATION CHANNEL NSX CREATES BETWEEN ESXI HOSTS, THIS CHANNEL IS PROVIDED BY VTEPS CREATED ON EACH ESXI HOST

VTEP interfaces • When a cluster is prepared a vmkernel interface (vtep) is added to every host in the cluster. VXLAN encapsulated packets are sent out on this interface, changes to this interface i.e. IP settings could cause network disruptions, these vmks should never be changed.

Teaming policy • The dvSwitch teaming policy is configured when VXLAN is configured on the dvSwitch. All vxlan portgroups will have the same teaming policy as the original settings during installation. Changing the team policy manually in vCenter can cause unpredictable issues.

MTU • VXLAN traffic needs at least 1600 MTU to configured on the dvSwitch due to the encapsulation. Lowering the MTU on the dvSwitch would cause VXLAN connectivity issues.