ESXi hosts trigger "Virtual machine creation might fail because the agent was unable to retrieve virtual machine creation options from the host."

Products

• VMware vSAN

Issue/Introduction

In a VMware vSphere environment, ESXi hosts intermittently trigger warning alarms regarding virtual machine creation capabilities.

Specifically, may observe the following symptoms:

• vCenter Server UI: "Virtual machine creation might fail because the agent was unable to retrieve virtual machine creation options from the host."
  
  

• Restarting the host management agents (hostd) or rebooting the affected ESXi hosts may temporarily resolves the issue and the alarm eventually recurs.

• From hostd, an error event logged regarding the 'vsantracesIODiag' file.
  
  2026-05-12T13:49:54.424Z In(166) Hostd[9377153]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 26742 : Issue detected on ABC in ha-datacenter: BC: 3042: File vsantracesIODiag--2026-05-12T13h34m38s827--#########-####-#####-####-##############.zst closed w (2026-05-12T13:49:54.422Z cpu65:2101820 opID=7025138a)


• And vmkwarning gives errors regarding "dirty buffer". However, this does not appear to be related to physical data loss.

  2026-05-15T01:35:39.658Z Wa(180) vmkwarning: cpu36:2097673)WARNING: Vol3: 4678: Error closing the volume: . Eviction fails: Failure
2026-05-15T01:42:18.589Z Al(177) vmkalert: cpu0:2101723 opID=50e0722a)ALERT: BC: 3042: File vsantracesIODiag--2026-05-15T01h28m51s684--#########-####-#####-####-##############.zst closed with dirty buffers. Possible data loss.

Environment

VMware vSphere ESXi 8.x
VMware vSAN 8.x

Cause

The issue occurs due to a known logging bug where the vsantrace object utilizes excessive local ramdisk space and fails to automatically expand or rotate properly.
When the vsantrace buffer cache cannot flush data to the disk, the underlying storage becomes exhausted. This triggers a "No space left on device" condition, which starves the host management agent (hostd) of the resources required to process and cache virtual machine creation options

From ESXi, var/run/log/vmkernel.log there is an error event with "No space left on device."

2026-05-15T01:43:02.956Z In(182) vmkernel: cpu2:2101723 opID=50e0722a)Fil3: 11066: Max no space retries (10) exceeded for caller Fil3_CreateAndOpenFile (status 'No space left on device')
2026-05-15T01:43:02.956Z In(182) vmkernel: cpu2:2101723 opID=50e0722a)Fil3: 397: numWanted: 25 numRCsPicked: 24 numNoSpace: 24 numLockNotFree: 14 numAllocFailed: 0 numSecondPass: 7 rcsWithNoValidRes: 0 rcsWithHighAffCnt: 0

2026-05-15T01:43:15.050Z In(182) vmkernel: cpu1:2101723 opID=50e0722a)ResAff: 13504: ########-####-####-####-##############: <FD c27 r129> - NSPC: lt 0 oo 10 nc 112 nvr 0 ac 0 dl 8 sp 0 pa 0 ma 0 tt 0 rjP 0 rjN 56:No space left on device
2026-05-15T01:43:15.050Z In(182) vmkernel: cpu1:2101723 opID=50e0722a)Fil3: 11066: Max no space retries (10) exceeded for caller Fil3_CreateAndOpenFile (status 'No space left on device')
2026-05-15T01:43:15.050Z In(182) vmkernel: cpu1:2101723 opID=50e0722a)Fil3: 397: numWanted: 25 numRCsPicked: 26 numNoSpace: 26 numLockNotFree: 16 numAllocFailed: 0 numSecondPass: 7 rcsWithNoValidRes: 0 rcsWithHighAffCnt: 0

vsantrace object details:
structtype:
  ObjectInfo
Health:
  healthy
Object UUID:
  ########-###-###-####-##############
Version:
  20
Owner:
  ESX04
Policy:
  stripeWidth: 1
Used:58380517376
Used 4K Blocks:57746243584
Size:
  30064771072
Type:
  vmnamespace
Path:
  /vmfs/volumes/vsan:######################-############/.vsan.trace (Exists)
Group UUID:
  ########-546a-253f-####-##############
Directory Name:
  .vsan.trace
Configuration:

      RAID_1
         Component: ########-####-####-####-#############
           Component State: ACTIVE,  Address Space(B): 30064771072 (28.00GB),  Disk UUID: 52588a0d-####-####-####-#############,  Disk Name: naa.###################:2
           Votes: 1,  Capacity Used(B): 29481762816 (27.46GB),  Physical Capacity Used(B): 29188161536 (27.18GB),  Total 4K Blocks Used(B): 28870983680 (26.89GB),  Host Name: ESX02
         Component: #######-####-####-####-##############
           Component State: ACTIVE,  Address Space(B): 30064771072 (28.00GB),  Disk UUID: 52af32cf-####-####-####-############,  Disk Name: naa.###################:2
           Votes: 1,  Capacity Used(B): 29481762816 (27.46GB),  Physical Capacity Used(B): 29188161536 (27.18GB),  Total 4K Blocks Used(B): 28871065600 (26.89GB),  Host Name: ESX01
      Witness: #########-####-####-####-################
        Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: 52d50d0d-####-####-####-############,  Disk Name: mpx.vmhba0:C0:T2:L0:2
        Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Total 4K Blocks Used(B): 4194304 (0.00GB),  Host Name: ESX03

Resolution

This is a known issue in ESXi 8.0 Update 3. To resolve the continuous logging bug and prevent the buffer cache from exhausting device space, upgrade the ESXi host to 8.0 Update 3h(25067014)