NSX Endpoint Security Multiplexer (EPSEC-MUX) Interfering with ESXi Host Upgrade Process
Article ID: 381495
Updated On: 12-12-2024
Products
Issue/Introduction
During the installation or upgrade of ESXi hosts to version 8.0, the following error message may be encountered:
These VIB(s) on the host do not have the required sha-256 checksum for their payloads: VMware_bootbank_epsec-mux_6.5.0esx60-7357581 This will prevent VIB security verification and secure boot from functioning properly. Please remove these VIBs and check with your vendor for a replacement of these VIBs.
Environment
VMware ESXi
VMware NSX
Cause
The EPSEC-MUX (Endpoint Security Multiplexer) module is not in use, as the NSX environment has not been set up. However, the outdated EPSEC-MUX VIB (version 6.5.0esx60-7357581) still resides on the ESXi host, causing conflicts during the upgrade process.
Resolution
1.Verify the presence of the outdated EPSEC-MUX VIB on this ESXi host:
- Log in to the ESXi host as root.Check if the EPSEC-MUX VIB is installed using the following command:
esxcli software vib list | grep epsec-mux
- Check the status of the vShield-Endpoint and NSX-Context-Engine services:
- Use the following command to check the service status:
/etc/init.d/vShield-Endpoint-Mux status - Example output:
[root@esxihost:/etc/init.d] ./vShield-Endpoint-Mux statusvShield-Endpoint-Mux is not runningNSX-Context-Engine is running
2. Remove outdate VIB using below command
esxcli software vib remove -n epsec-mux
If removing VIB failed and throws error as below:
3. The stop the vShield-Endpoint-Mux and NSX-Context-Engine services if they are running:
- To stop the vShield-Endpoint-Mux service, run:
/etc/init.d/vShield-Endpoint-Mux stop
- To stop the NSX-Context-Engine service, use:
/etc/init.d/vShield-Endpoint-Mux stop-nsx-ctxteng
4 .If the services cannot be stopped, check if the following daemons are enabled at the runlevel:
- Run the following command to check:
chkconfig --list
- If watchdog-vShield-Endpoint-Mux and watchdog-NSX-Context-Engine daemons are enabled (or if only the vShield-Endpoint-Mux daemon is enabled), turn them off using the following command:
chkconfig vShield-Endpoint-Mux off
5. And then Stop the vShield-Endpoint-Mux and NSX-Context-Engine services again:
- After disabling the daemons, this will allow to stop these services now:
/etc/init.d/vShield-Endpoint-Mux stop /etc/init.d/vShield-Endpoint-Mux stop-nsx-ctxteng
6. Additionally, check if any Mux processes are currently running:
- Run the following command to verify if the vShield-Endpoint-Mux processes are active on the ESXi host:
# ps -c | grep Mux 192223 192223 sh /bin/sh /sbin/watchdog.sh -s vShield-Endpoint-Mux -q 100 -t 1000000 /usr/lib/vmware/vShield-Endpoint-Mux 900 -c 910 192233 192233 vShield-Endpoint-Mux /usr/lib/vmware/vShield-Endpoint-Mux 900 -c 910 192236 192233 vShield-Endpoint-Mux /usr/lib/vmware/vShield-Endpoint-Mux 900 -c 910
If any Mux processes are running, use the following command to kill the processes (including the watchdog.sh process):
# kill -9 192223 192233 192236
7. Once the services are stopped, remove the outdated EPSEC-MUX VIB using the following command:
esxcli software vib remove -f -n epsec-mux --no-live-install
8. After the VIB is removed, confirm that it's no longer present on the ESXi host:
esxcli software vib list | grep epsec-mux
[Note: All these steps on the ESXi host command shell with root login.]
Additional Information
In this case, since NSX is not being used, stopping the vShield-Endpoint-Mux service and removing the EPSEC-MUX VIB is not an issue. However, for customers who are actively using NSX, may require further investigation on obtaining a replacement EPSEC-MUX VIB with the correct checksum .
Kindly Ref below docs for more details:
Troubleshooting ESX GI Module (MUX)
Collecting diagnostic information for the NSX Guest Introspection MUX VIB
Feedback
