Host Remediaton fails intermittently when upgrading ESXi version with single image managed cluster

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:
lifecycle.log reports that the remediation failed due to "No space left on device", while vmkernel.log shows message like:
Admission failure in path: "host/vim/vmvisor/settingsd-task-forks/python.<PID>:python. <PID>:uw. <PID>"

Environment

VMware vCenter Server 7.0.3
VMware vSphere ESXi 7.0.3

Cause

The settings memory pool (used by VLCM image cluster management) gets full during the remediation process while the host still has memory available. During the remediation, the self-imposed memory limit exceeds when a separate process is forked to update the bootloader towards the end of the remediation.

Resolution

To resolve the issue follow one of the below mentioned approches:

1. Reboot the host and retry remediation. If any issues occur, please rollback to the previous ESXi version and try one of the approaches below.
2. The fixes will be made available in future releases. Once available, remediate the host to an ESXi version that contains the fix for the issue.
3. If feasible, move hosts back to a baseline management cluster.
4. If feasible, use esxcli command to manually perform the upgrade.