No space left on device while upgrading ESXi with esxcli

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Can not upgrade ESXi since "No space left on device"
• Sample logs:
  • $  esxcli software vib update -d "/vmfs/volumes/DATASTORE/A.zip"
    [InstallationError]
    [Errno 28] No space left on device
           vibs = VMware_bootbank_esx-base_xxx
    Please refer to the log file for more details.
  • In vob.log
     [vob.visorfs.ramdisk.full] Cannot extend visorfs file /tmp/stagebootbank/weaselin.t00 because its ramdisk (stagebootbank) is full.
    [esx.problem.visorfs.ramdisk.full] The ramdisk 'stagebootbank' is full. As a result, the file /tmp/stagebootbank/weaselin.t00 could not be written.
  • In vmkernel.log

            WARNING: VisorFSRam: 351: Cannot extend visorfs file /tmp/stagebootbank/weaselin.t00 because its ramdisk (stagebootbank) is full.
            BC: 4992: Failed to flush 121 buffers of size 8192 each for object 'weaselin.t00' 1 8 1a078 3ed2fae0 0 0 0 0 0 0 0 0 0 0: No free memory for file data

• With df -h, the partition is almost full.

vfat 261853184 258146304 3706880 99% /vmfs/volumes/########-#######-#####-#########
vfat 261853184 249724928 12128256 95% /vmfs/volumes/########-#######-#####-#########

• use vdf -h and compare with other normal host, you may find some additional vibs was installed on the issue host.

Environment

VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7

Cause

There is no space available in RAM to install the update.

Resolution

Remove those additional big vib and retry again.