Having host error issues when adding a host to the NSXT platform

Products

• VMware NSX

Issue/Introduction

The host was previously connected to NSX-V and was removed and an attempt to add it to NSX-T shows the following error in the UI:

Failed to install software on host. XXX-XXX-esxXXX.XX.XXXXXXXXXX.com : java.rmi.RemoteException: [LiveInstallationError] Error in running [/etc/vmware/shutdown/shutdown.d/vShield-Endpoint-Mux]: Return code: 1 Output: rm: can't remove '/usr/lib/vmware/vsepmux/bin/vShield-Endpoint-Mux': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/bin/namespace_db.py': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/bin/NSX-Context-Engine': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/bin': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/etc/rdsh.conf': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/etc': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/lib64/libprotobuf.so.9.0.0': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/lib64/libboost_system-gcc41-mt-1_42.so.1.42.0': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/lib64/libboost_system-gcc41-mt-1_42.so': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/lib64/libboost_thread-gcc41-mt-1_42.so': Operation not permitted rm: can't remove '/usr/lib/vmware/vsepmux/lib64/libboost_thread-gcc41-mt-1_42.so.1.42.0':

 

When you run  grep "vShield-Endpoint-Mux" from the /commands directory on the ESXi host you may see the following output::

 key = 'vShield-Endpoint-Mux',
 label = 'vShield-Endpoint-Mux',
key = 'vShield-Endpoint-Mux-Partners',
label = 'vShield-Endpoint-Mux-Partners',
key = 'vShield-Endpoint-Mux',
label = 'vShield-Endpoint-Mux',
key = 'vShield-Endpoint-Mux-Partners',
label = 'vShield-Endpoint-Mux-Partners',

 

Environment

NSX-T

Cause

NSX-V was not properly removed / uninstalled when the host was removed from NSX-V

Resolution

vShield needs to be removed so the host can be added to NSX-T

 

Verify that vShield is running from SSH to the ESXI host as ROOT:

[root@cld-nsh1-esxXXX:~]  /etc/init.d/vShield-Endpoint-Mux status
vShield-Endpoint-Mux is not running
NSX-Context-Engine is running

Delete the vShield VIB

[root@cld-nsh1-esxXXX:~] esxcli software vib remove -n=epsec-mux --no-live-install

You should see the following post removal

Removal Result
  Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
  Reboot Required: true
  VIBs Installed:
  VIBs Removed: VMware_bootbank_epsec-mux_6.5.0esx60-8462817
  VIBs Skipped:

 

After the reboot you can add the host to NSX-T