ESXi boot failures due to system configuration issues - restore security configuration, decrypt system configuration, recover system configuration

Products

• VMware vSphere ESXi

Issue/Introduction

This article will assist when troubleshooting the failure of an ESXi host to boot after upgrade/installation to vSphere 7.0 U2 or later. The article aims to help eliminate the common causes of this issue by verifying the minimum system requirements are met and the hardware is functioning as expected.

Possible Error Messages

• The system has found a problem on your machine and cannot continue.
• Unable to restore the system configuration. A security violation was detected. https://via.vmw.com/security-violation



• Failed to decrypt system configuration. https://via.vmw.com/config-decryption-failed



• Unable to recover the system configuration. https://via.vmw.com/recovery-failed
  
  

Note: Before ESX 8.0 U1, Quick Boot cannot be used when TPM is enabled.

Environment

VMware vSphere ESXi 8.x
VMware vSphere ESXi 7.x

Resolution

Validate that each step below is true in the environment. Each step provides instructions or a link to a document to eliminate possible causes and take corrective action as necessary.

Error: "Unable to restore the system configuration. A security violation was detected. https://via.vmw.com/security-violation"

1. Make sure that the configuration abides by the configuration requirements:
   
   • Refer to TPM requirements: https://knowledge.broadcom.com/external/article/312159/vmware-vsphere-support-of-trusted-platfo.html
   • Make sure the TPM device is detected and healthy from the iDRAC.
   • If you are using TPM 2.0, Ensure that UEFI, TPM, and Secure Boot are enabled. If you are using TPM 1.0, secure boot is not required.
   • Refer to your host's documentation for setting up the BIOS for required settings.
     
     
2. Check if the firmware security settings have been modified from previous. You can refer to another host in your environment if your configuration is uniform accross hosts. 
   • If TPM 2.0 has been disabled, re-enable it.
   • If UEFI secure boot has been disabled, enable it.
   • If execInstalledOnly boot option is set to FALSE, change it back to its initial value (i.e. TRUE).
   • If execInstalledOnly has been disabled, add "execInstalledOnly=TRUE" to the boot command-line (press shift+o when mboot starts and can see a 5 second countdown, right after the bios finishes running).
     
     
3. To change the firmware settings and permanently avoid this violation message, refer to Enable or Disable the Secure Boot Enforcement for a Secure ESXi Configuration.
   
   
4. If the firmware settings have not been modified, this means that either the TPM 2.0 chip is not working or has been replaced (possibly due to a system board change) or the version of ESXi being booted is not genuine. In this case, recover the ESXi configuration with the following steps: 
   • Start the ESXi host.
   • When the ESXi installer window appears, press 'Shift+O' to edit boot options.
   • To recover the configuration, at the edit boot options, append the following encryption recovery key:
     
     encryptionRecoveryKey=######-######-######-######-######-######-######-######-######-######-######-######-######
     
     Example:
     
     
     Note:
     • Do not remove the information which is already present at the prompt. At the end of the existing bootUUID, add a space followed by the encryptionRecoveryKey information as shown in the above screenshot.
     • The recovery key can be obtained by executing the command "esxcli system settings encryption recovery list" while the ESXi host is healthy. The key consists of 16 sets of six digit values, with a dash between each set.
     • After obtaining this key, it should be recorded and securely stored. However, if the recovery key is not available, the only option is to reinstall ESXi - refer to Install and upgrade ESXi step by step procedures.
       
       
   • Press enter to continue the host boot process.
     • Now, the secure ESXi configuration is recovered and the ESXi host boots. 
     • To persist the change, enter the following command:
       /sbin/auto-backup.sh
       
       
   • Reboot the ESXi host.
   • If after rebooting the ESXi host and all above checks in this article have been confirmed, the PSOD repeats with "Unable to restore the system configuration. A security violation was detected", then some of the issues below may have occured. Contact your hardware vendor to investigate the hardware. Note that this is not an all inclusive list but lists some possible causes:
     • The TPM chip is broken/data is corrupted, hence writing to memory fails.
     • TPM chip seating problem
     • Motherboard issues

Note: For ESXi versions 8.0 U1 and 8.0 U2 (or any patch on these lines), if a PSOD is encountered after an ESXi quick boot upgrade, simply rebooting the host will solve the problem. VMware is aware of this issue and working on a fix in a future release.

Error: Failed to decrypt system configuration. https://via.vmw.com/config-decryption-failed

This means that a genuine ESXi version has booted, but the configuration data has been tampered with or is corrupted and cannot be recovered. Refer to Install and upgrade ESXi step by step procedures.

Error: Unable to recover the system configuration. https://via.vmw.com/recovery-failed

This means that ESXi is unable to be recovered with the provided recovery key. Ensure the input recovery key is correct; otherwise, refer to Install and upgrade ESXi step by step procedures.