ESXi host remediation fails with "Cannot copy files to host data.host" when remediated using LCM baseline.

Products

• VMware vSphere ESXi

Issue/Introduction

• Patching ESXi host using vSphere Lifecycle Manager baselines fails.
• The baseline remediation failed at 'scan entity' task with below error message:
  "Cannot copy files to host data.host. Check the management service status, network connectivity with vCenter and disk space on the host."
• There are no space issues on the ESXi host in any partition.
• All the management services are running fine on ESXi host.
• Running vdf -h command will show upgradescratch partition 
  

  Ramdisk                   Size      Used Available Use% Mounted on

  upgradescratch            360M        0B      360M   0% --

• vCenter vum-server logs reports below error: "Com.vmware.vcIntegrity.hostUpgradeNfcCopyFailure"
  

• vmkernel.log of the Host shows below error:
  "Admission failure in path host/system/kernel/upgradescratch:upgradescratch"
  
  
• vmkernel.log may also report "Cannot extend visorfs file /upgrade_scratch/SB.V00 because its ramdisk (upgradescratch) is full"

Environment

VMware vSphere ESXi 7.0

VMware vSphere ESXi 8.0

VMware vCenter Server 7.0

VMware vCenter Server 8.0

Cause

The issue arises when multiple upgrade baselines are present. When you remediate one baseline, all baselines are scanned.
When scan happens it also creates upgradescratch ramdisk on ESXi, so when multiple scan happens the size of the ramdisk depends on the last one scanned which is not necessarily the one being remediated. Hence Lifecycle Manager checks the disk space on host.

Resolution

Please take a snapshot of the vCenter VM before performing the following steps. In case of Enhanced Linked mode, consider taking offline snapshots:

• Check the list of baselines that are attached to the cluster or ESXi host.
• Detach unnecessary baselines attached to the host which were previously used to update\upgrade hosts.
• Navigate to Lifecycle manager and delete the older baselines apart from predefined. Also, delete older ISOs which were imported in imported ISOs section.