An error occurred during host configuration: /usr/sbin/esxupdate returned with exit status: 15
search cancel

An error occurred during host configuration: /usr/sbin/esxupdate returned with exit status: 15

book

Article ID: 373343

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • Users may encounter failures when attempting to upgrade ESXi 7.0 to a newer build. The upgrade process may fail with timeout errors or "No space left on device" messages, despite apparent available disk space on the host.
  • When trying to update ESXI, the error below is displayed:
 [InstallationError]
 VMware_locker_tools-light_12.5.1.24649672-24723868: VIB tools-light requires 227 MB free space in the locker partition to be safely installed, however the parition only has 205 MB available. Please clean up the locker partition and retry the transaction.
      cause = VMware_locker_tools-light_12.5.1.24649672-24723868: VIB tools-light requires 227 MB free space in the locker partition to be safely installed, however the parition only has 205 MB available. Please clean up the locker partition and retry the transaction.
       vibs = ['VMware_locker_tools-light_12.5.1.24649672-24723868']
 Please refer to the log file for more details.
  • You see log entries, such as:  From /var/run/log/esxupdate.log:
esxupdate: 2128275: LockerInstaller: DEBUG: Locker partition has 23798480896 bytes of free space
esxupdate: 2128275: LockerInstaller: DEBUG: Payload tools of VIB VMware_locker_tools-light_12.3.5.22544099-23794019 requires        191837891 bytes after extraction
esxupdate: 2128275: root: ERROR: Timeout (30 seconds) expired waiting for output from command '['/bin/tar', 'xzf', '-', '-C', '/locker/packages/']', pid '2128316'.

 OR 

  • You see log entries as below: From /var/run/log/hostd.log:

YYYY-MM-DDT In(166) Hostd[2098519]: -->  eventTypeId = "esx.problem.esximage.install.stage.error",

YYYY-MM-DDT In(166) Hostd[2098519]: -->  key = "1",

YYYY-MM-DDT In(166) Hostd[2098519]: -->  value = "VMware_locker_tools-light_12.5.3.24819442-24853260: Error untarring to /locker/packages/, return code: 1

YYYY-MM-DDT In(166) Hostd[2098519]: -->     stdout: b''

YYYY-MM-DDT In(166) Hostd[2098519]: --> stderr: b'tar: write error: No space left on device\n'"

YYYY-MM-DDT In(166) Hostd[2098546]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 144 : Could not stage image profile '': VMware_locker_tools-light_12.5.3.24819442-24853260: Error untarring to /locker/packages/, return code: 1

Disk space check (df -h) shows less space available

Filesystem   Size   Used   Available   Use%   Mounted on
 VMFS-L       23.8G  1.4G   205 MB     /vmfs/volumes/LOCKER-xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxxx

Additional Information :

1. Environments running Nutanix Clusters, where Nutanix LCM UI is used to upgrade the ESXi Host to the latest version may run into the same issue.

2. The Error seen in the Nutanix LCM UI is as below with the Error "Vib Installation Failed" and the esxupdate.log shows that the Error "227 MB free space in the locker partition to be safely installed, however the partition only has 205 MB available. Please clean up the locker partition and retry the transaction."

3. When the same upgrade is attempted via vCenter LCM the ESXi Host upgrade completes successfully without any issues

4. The insufficient space in the /Locker and additional files / Vibs being pushed from Nutanix is triggering this behavior, the fix is to follow the below resolution section 
NOTE - Even with 1GB of free space Nutanix LCM UI was failing to upgrade the Hosts and freeing up more space in the /Locker partition helps to resolve this issue.

 

Environment

ESXi 7.x

ESXi 8.x

Cause

The primary cause of this issue is insufficient space in the /locker partition, which is used during the ESXi upgrade process. This can occur even when the overall host appears to have adequate free space, as the /locker partition is separate from the main datastore.

Resolution

1. Check /locker partition space:
   a. SSH into the ESXi host.
   b. Run the command: df -h /locker
   c. Ensure there is at least 1GB of free space.

2. If space is low, clean up the /locker/packages directory:
   a. Run: rm -rf /locker/packages/*
   b. Verify with: ls -la /locker/packages

3. Examine /locker usage:
   a. Run: du -sh /locker/*
   b. Identify and remove any unnecessary large files or directories.

4. Verify permissions on /locker/packages:
   a. Run: ls -ld /locker/packages
   b. Ensure permissions are set to 755 (drwxr-xr-x).
   c. If needed, run: chmod 755 /locker/packages

5. Clear temporary files in ramdisk:
   a. Run: rm -rf /tmp/*
   b. Run: rm -rf /var/tmp/*

6. Attempt the upgrade again using your preferred method (VUM, esxcli, etc.).

7. If the issue persists, review ESXi logs:
   a. Run: tail -n 100 /var/log/vmkernel.log
   b. Run: tail -n 100 /var/log/esxupdate.log
   c. Look for errors related to space issues or /locker.

8. If problems continue, consider performing an offline upgrade using the ESXi ISO.

 

Additional Information

Powered by Wolken Software