Applying an NSX or ESXi upgrade bundle to a VDI workload domain fails at entering maintenance mode

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

1. LCM pre-checks on a VDI deployed Workload Domain fails on the dry-run for host maintenance mode.
2. NSX and ESXi bundle upgrades on the VDI workload domain with horizon instant clones, will fails at entering maintenance mode on the hosts 

Environment

VMware Cloud Foundation 3.9.x

Cause

This is an expected behaviour with Horizon and highlighted in https://kb.vmware.com/s/article/2144808 . Unfortunately, the mentioned KB only provides steps at a host level which can lead to further disruptions in the LCM upgrade process.

Resolution

This is a known issue, with currently no resolution. Please refer to steps below to workaround this issue.

Workaround:

1. Connect to Connection Server VM and run the command below to delete the parent VMs and prepare the hosts to enter maintenance mode:

Note: The command must be running against all VDI participating ESXi hosts
 

Example: 4 hosts are participating.

IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-1 host name> -maintenance ON
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-2 host name> -maintenance ON
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-3 host name> -maintenance ON
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-4 host name> -maintenance ON

2. Wait for approximately 3-5 minutes for the parent VMs to be deleted which ensures the maintenance activity will continue without any issues.
3. Navigate to SDDC Manager to schedule the appropriate upgrade bundle.
4. Wait for the upgrade report as successful.
5. Connect again to Connection Server VM and take the hosts out of maintenance mode.

IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-1 host name> -maintenance OFF
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-2 host name> -maintenance OFF
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-3 host name> -maintenance OFF
IcMaint.cmd -vc <VC host name or IP> -uid <user ID> -password <user password> -hostName <ESXi-4 host name> -maintenance OFF

6.  Open a browser to Connection sever URL, and select Desktop Pool.
7. Select "Enable provisioning" from the respective pool options to recreate the parent VMs on respective hosts.

Additional Information

Impact/Risks:
Warning: Desktop provisioning will be blocked during this workaround. Please make sure not to log off / sign out from the desktops during the maintenance window to ensure the desktops stay up and running as instant clones that are logged off will be deleted or refreshed as the default behaviour.