Host reports lost communication with ESX module when using instant clone

Products

• VMware NSX for vSphere

Issue/Introduction

Symptoms:

• After deploying a Guest Introspection on an ESXi host with Instant Clone VMs running the Thin agent, the Guest Introspection service goes into Warning status on the Service Deployment.
   
• On the host, under Monitor > Guest Introspection, you see an error similar to:
  Lost communication with ESX module.

Environment

VMware NSX for vSphere 6.3.x
VMware NSX for vSphere 6.2.x

Cause

This issue occurs when the MUX blocks the call to a quiesced virtual machine.When using Instant clone virtual machines in View, the parent/main image is permanently quiesced, which causes the MUX to block as soon as a connection is made from this quiesced virtual machine.

Resolution

This issue is resolved in:

• ESXi 6.0 P05 (ESXi600-201706001). 
• ESXi 6.5 U1 (ESXi650-201707201). 

To work around this issue, remove all the View parent virtual machines from the ESXi host.

 

1. Select the host that you want to put in maintenance mode.
   If you are using the vSphere web client, ensure that the plug-in to edit Annotations is installed.
2. Search for annotations in the host's Summary tab and set the InstantClone.Maintenance value to 1.
3. After 3 minutes all parent virtual machines on the host are deleted. Also, the InstantClone.Maintenance value changes to 2.
4. Clear the InstantClone.Maintenance annotation value.

Note: After removing all View parent virtual machines, the EPsec MUX service will be restored automatically without any further action.

For more information, see Entering and exiting maintenance mode for an ESXi host that has Horizon instant clones (2144808)

 

To avoid this issue:

1. Stop the Guest introspection service on ESXi hosts by running this command:
   /etc/init.d/vShield-Endpoint-Mux stop
    
2. Complete the view pool provisioning/ deploy of new parent virtual machine.
3. After deploying all virtual machines, start the vShield-Endpoint-Mux service by running this command:
   /etc/init.d/vShield-Endpoint-Mux start.
    
4. You can see the normal status on the Solutions deployment page.

Additional Information