Deploying VMware NSX for vSphere 6.x through Auto Deploy

search cancel

Deploying VMware NSX for vSphere 6.x through Auto Deploy

book

Article ID: 327400

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

This article provides steps to prepare ESXi hosts in vSphere clusters as part of Auto Deploy configurations in an NSX for vSphere 6.x

Environment

VMware NSX for vSphere 6.3.x
VMware NSX for vSphere 6.1.x
VMware NSX for vSphere 6.2.x
VMware NSX for vSphere 6.0.x
VMware NSX for vSphere 6.4.x

Resolution

Requirements:

Before proceeding, ensure to:

Auto deploy infrastructure
VXLAN infrastructure (NSX Manager and a DHCP server)
Windows host has PowerCLI installed
Copy of the ESXi offline depot. For example, VMware-ESXi-5.1.0-799733-depot.zip
Copy of an additional vxlan offline depot. For example, vxlan.zip.

You can download this by connecting to the NSX Manager at:

https://vsm-ip/bin/vdn/vibs/5.5/vxlan.zip

Notes:
- Prior to NSX 6.2.0, the vxlan.zip contains 3 VIBs, esx-vxlan, esx-vsip and esx-dvfilter-switch-security. In NSX 6.2.x and 6.3.x, the vxlan.zip contains only 2 VIBs, esx-vxlan and esx-vsip. The contents of the esx-dvfilter-switch-security and the new traceflow VIBs were folded into the esx-vxlan VIB.
- Also prior to NSX 6.2.0, there was a single URL on NSX Manager from which VIBs for a certain version of the ESXi Host could be found. In other words, the administrator only needed to know a single URL, regardless of NSX version.
- Starting in NSX 6.2.4, a third VIB, esx-vdpi, is provided along with the esx-vsip and esx-vxlan NSX VIBs.
- In NSX 6.4.x, the vxlan.zip file contains a single VIB file "VMware_bootbank_esx-nsxv.vib".
  Example: For ESXi 6.5, the name of the VIB file is "VMware_bootbank_esx-nsxv_6.5.0-0.0.13168956.vib".
- In NSX 6.2.x, 6.3.x, and 6.4.x, the NSX VIBs are available at different URLs. To find the correct VIBs, find the new VIB URL from https://NSX_Manager_IPaddress/bin/vdn/nwfabric.properties.The nwfabric.properties returns the correct download paths based on versions:
For NSX for vSphere 6.2.x
# 5.1 VDN EAM Info
VDN_VIB_PATH.1=/bin/vdn/vibs-6.2.0/5.1-2107743/vxlan.zip
VDN_VIB_VERSION.1=2107743
VDN_HOST_PRODUCT_LINE.1=embeddedEsx
VDN_HOST_VERSION.1=5.1.*

# 5.5 VDN EAM Info
VDN_VIB_PATH.2=/bin/vdn/vibs-6.2.0/5.5-2983935/vxlan.zip
VDN_VIB_VERSION.2=2983935
VDN_HOST_PRODUCT_LINE.2=embeddedEsx
VDN_HOST_VERSION.2=5.5.*

# 6.0 VDN EAM Info
VDN_VIB_PATH.3=/bin/vdn/vibs-6.2.0/6.0-2984108/vxlan.zip
VDN_VIB_VERSION.3=2984108
VDN_HOST_PRODUCT_LINE.3=embeddedEsx
VDN_HOST_VERSION.3=6.0.*
For NSX for vSphere 6.3.x
# 5.5 VDN EAM Info
VDN_VIB_PATH.1=/bin/vdn/vibs-6.3.0/5.5-4744075/vxlan.zip
VDN_VIB_VERSION.1=4744075
VDN_HOST_PRODUCT_LINE.1=embeddedEsx
VDN_HOST_VERSION.1=5.5.*

# 6.0 VDN EAM Info
VDN_VIB_PATH.2=/bin/vdn/vibs-6.3.0/6.0-4744062/vxlan.zip
VDN_VIB_VERSION.2=4744062
VDN_HOST_PRODUCT_LINE.2=embeddedEsx
VDN_HOST_VERSION.2=6.0.*

# 6.5 VDN EAM Info
VDN_VIB_PATH.3=/bin/vdn/vibs-6.3.0/6.5-4744074/vxlan.zip
VDN_VIB_VERSION.3=4744074
VDN_HOST_PRODUCT_LINE.3=embeddedEsx
VDN_HOST_VERSION.3=6.5.*

Steps to prepare ESXi hosts for NSX as part of Auto Deploy

To include VXLAN as a part of Auto Deploy:

Create a new image profile
Update Auto Deploy rules
Configure VXLAN
Update the host profiles

Creating a new image profile

To create a new image profile:

Start PowerCLI and connect to vCenter Server by running this command:

Connect-VIServer #.#.#.# –User user -Password password
Import offline depots by running these commands:

C:\Add-EsxSoftwareDepot C:\path\VMware-ESXi-5.1.0-799733-depot.zip
C:\Add-EsxSoftwareDepot C:\path\vxlan.zip

Note: Repeat these commands for all depots
Run this command to list the default Image Profiles and note the name of profile to be cloned, such as ESXi-5.1.0-799733-no-tools:

C:\Get-EsxImageProfile
Run this command to create new Image Profile by cloning an existing image profile, such as ESXi-5.1.0-799733-no-tools:

C:\New-EsxImageProfile -Name NewImageProfile -CloneProfile ESXi-5.1.0-799733-no-tools VMware –AcceptanceLevel PartnerSupported
For NSX 6.3.x, run this command to add the two required NSX for vSphere VIBS to the new Image Profile. Ensure that the Image Profile includes both VIBs, or you may experience unexpected results.

C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vxlan
C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vsip

Note: Starting in NSX for vSphere 6.2.4, a third VIB, esx-vdpi, is provided along with the esx-vsip and esx-vxlan NSX VIBs. Also note that this does not apply to NSX for vSphere 6.3.x.

C:\Add-EsxSoftwarePackage -ImageProfile NewImageProfile -SoftwarePackage esx-vdpi
Run this command to list all VIBs in the new Image Profile:

C:\Get-EsxImageProfile NewImageProfile | Select -ExpandProperty VibList | sort
(Optional) Run this command to save the new Image Profile as an offline depot:

C:\ESXi-Depots> Export-EsxImageProfile -ImageProfile NewImageProfile -ExportToBundle -FilePath C:\path\FileName.zip

Updating Auto Deploy rules

To update the Auto Deploy rules:

Run this command to list the current rules:

C:\> Get-DeployRuleSet
Run this command to remove the old rule:

C:\> Remove-DeployRule old_rule

Note: This command removes the rule, but does not delete it.
Create a new rule for the new Image Profile.

For example, run this command to create the rule CustomImageRule using the NewImageProfile created earlier and to assign it to all hosts that boot using Auto Deploy.

C:\> New-DeployRule CustomImageRule -Item NewImageProfile –AllHosts
Run this command to activate the new rule:

C:\> Add-DeployRule CustomImageRule
Run these commands to update the existing hosts to use the new rule:

C:\> foreach ($esx in Get-VMHost) {$esx | test-deployrulesetcompliance | Repair-DeployRuleSetCompliance}
C:\> foreach ($esx in Get-VMHost) {$esx | test-deployrulesetcompliance}

Note: The changes take affect after the next reboot.

Configuring VXLAN
Note: This step is not required if there is no need for VXLAN overlay services in the environment.

If this is a new NSX cluster, open NSX Manager and install the Network Virtualization Components on all hosts in the NSX cluster.

To prepare the hosts for VXLAN, configure these parameters in the preparation workflow:

Choose the cluster and the vSphere Distributed Switch (vDS)
Assign the VLAN for VXLAN transport traffic
Select the Teaming policy and MTU

Updating the host profiles
Note: This step is always required, even if the hosts will not be prepared for VXLAN.

To avoid unexpected results, ensure that the initial host profile used before NSX preparation is created after the ESXi host has booted with an image profile containing the NSX VIBs. This order of operation ensures that the host profile contains the VXLAN netstack and is not removed when the profile is applied.

Note: vSphere releases which support VXLAN use separate TCP/IP stack or netstack instances to provide isolation and abstraction. The esxcli network ip netstack get command retrieves the netstacks - defaultTCPipStack and VXLAN.

After the hosts are prepared for NSX and VXLAN is configured, update the host profile from the reference host and remove certain host-specific parameters.

To update the host profile from the reference host and remove certain host-specific parameters:

Right-click the host profile and click Enable/Disable Profile Configuration.
Under Networking configuration > Host Virtual NIC, deselect the VXLAN vmkernel interface to ensure that NSX Manager always creates the VTEP on boot.

Note: if your reference profile contains multiple VTEPs, ensure that all such VTEPs are removed.
Under Networking configuration > NetStack Instance > VXLAN, deselect the ipRouteConfig option. (If vSphere 6.5 version)
Under Networking configuration > NetStack Instance > VXLAN, ensure the ipRouteConfig option is selected. (if vSphere 6.7 version)
Under Advanced configuration option, deselect all UserVars.Rmq* variables. These variables are host-specific.

# grep Rmq /etc/vmware/esx.conf
/advUserOptions/options[0030]/name = "RmqClientPeerName"
/advUserOptions/options[0031]/name = "RmqHostId"
/advUserOptions/options[0036]/name = "RmqClientResponseQueue"
/advUserOptions/options[0037]/name = "RmqClientExchange"

With these changes you can refresh the host profile as required, and the NSX parameters are ignored when the host profile is applied.

Note: When setting the hostname, apply the default value to all netstack instances except for the default TCP/IP stack. To do so, VMware recommends Networking configuration > NetStack Instance > VXLAN > DNS Configuration > Hostname, select Obtain hostname from DCHP.

Note: When configuring a host for VXLAN in the NSX Manager UI, ensure that the configured LACP setting matches the setting on the virtual switch. If the settings differ, in the NSX Manager logs, you see entries similar to:

<YYYY-MM-DD>T<time>.650 GMT WARN http-443-exec-22 DefaultExceptionLogger:35 - The following exception occurred during request processing by the BlazeDS MessageBroker and will be serialized back to the client: </time>

flex.messaging.MessageException: Invalid LACP version selected. Distributed Virtual Switch vDS01 is configured with different LACP version.

Unpreparing a Host for VXLAN

To unprepare a host for VXLAN, you must remove the VXLAN network configuration parameters (IP route config and VTEP interfaces) as well as remove the three NSX VIBs from the image profile and create a new image profile. If the VIBs are not removed from the image profile, the host will show as Not Ready in the Host Preparation tab after reboot. Clicking on the red error next to the Not Ready link displays the reason for the error.

Additional Information

Feedback

thumb_up Yes

thumb_down No