ESXi Remediation appears to fail with Failed to update bootloader: [Errno 28] No space left on device
search cancel

ESXi Remediation appears to fail with Failed to update bootloader: [Errno 28] No space left on device

book

Article ID: 311953

calendar_today

Updated On: 03-04-2025

Products

VMware vSphere ESXi

Issue/Introduction

While upgrading ESXi from 7.0U2a to a U3 variant with the below addons, remediation appears to fails with internal error while staging/remediating the host.

On vCenter 7.0U3c, remediation fails with the below vendors:

  1. HPE (702.0.0.10.7.0-52 to 703.0.0.11.5.0-6)
  2. Cisco (4.1.3-a to 4.2.1-a)
  3. Hitachi (7.0.2a.13-01 to 7.0.3.14-03)

On vCenter 8.0U1, remediation fails with the below vendors:

  1. Inspur (7.0.2-2 to 7.0.3-3)
  2. Dell (A07 to A19)
  3. HPE (702.0.0.10.7.0-52 to 703.0.0.11.5.0-6)


ESXi logs - /var/log/lifecycle 

YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: Ramdisk:176 INFO Unmounting manual tardisk /tardisks.noauto/loadesx-135565
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:147 ERROR [InstallationError] VMware_bootbank_esx-base_7.0.3-0.20.19193900: Failed to update bootloader: [Errno 28] No space left on device
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR Traceback (most recent call last):
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1401, in _SysStorageBootPartAction
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/usr/lib/vmware/esxupdate/systemStorage.zip/systemStorage/esxboot.py", line 227, in installBootloader
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/usr/lib/vmware/esxupdate/systemStorage.zip/systemStorage/esxboot.py", line 98, in installBiosBootloader
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/usr/lib/vmware/esxupdate/systemStorage.zip/systemStorage/vfat.py", line 795, in mcopy
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/usr/lib/vmware/esxupdate/systemStorage.zip/systemStorage/vfat.py", line 765, in _mtools
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/lib64/python3.8/subprocess.py", line 489, in run
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/lib64/python3.8/subprocess.py", line 854, in __init__
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/lib64/python3.8/subprocess.py", line 1637, in _execute_child
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR OSError: [Errno 28] No space left on device
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR During handling of the above exception, another exception occurred:
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR Traceback (most recent call last):
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/usr/lib/vmware/lifecycle/bin/imagemanagerctl.py", line 636, in software
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR t.ApplyImage(swSpec, opts.depots, opts.taskId)
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 1891, in ApplyImage
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR res = NewTransaction.Transaction().PerformImageApply(newProfile,
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 1995, in PerformImageApply
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 1252, in _applyImageProfile
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 1398, in _validateAndInstallProfile
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/HostImage.py", line 1195, in Remediate
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1679, in Remediate
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1461, in _UpdateBootloader
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR File "/tmp/esx-update-135565/lib64/python3.8/site-packages/vmware/esximage/Installer/BootBankInstaller.py", line 1404, in _SysStorageBootPartAction
YYYY-MM-DDTHH:MM:SSZ lifecycle: 135565: imagemanagerctl:152 ERROR esximage.Errors.InstallationError: VMware_bootbank_esx-base_7.0.3-0.20.19193900: Failed to update bootloader: [Errno 28] No space left on device

In vCenter UI under events for the ESXI host an event similar to this will be found:

Date Time:
 MM/DD/YYYY, HH:MM:SS PM
Type:
 Error
Target:
 <ESXi Target Name>
Description:
Ask VMware...
Could not install image profile: VMware_bootbank_esx-base_7.0.3-0.55.20328353: Failed to update bootloader: [Errno 28] No space left on device
Related events:
There are no related events.

 

Environment

7.x
8.x

Cause

This is due to a resource pool limitation built into the U2a release.

Resolution

Currently, there is no resolution, upgrade to U3 is required

Workaround:

Manually reboot the host, once rebooted the updated image will be present upon boot thus completing the upgrade.

Powered by Wolken Software