vSAN ESA PSOD with "Failed to wait for object exit" message.
search cancel

vSAN ESA PSOD with "Failed to wait for object exit" message.

book

Article ID: 392796

calendar_today

Updated On:

Products

VMware vSAN 8.x

Issue/Introduction

PSOD display screen of ESXi host may begin with the following panic details:

"VMware ESXi 8.0.3 [Releasebuild-build-24280767 x86_64]
########-####-####-####-############:Failed to wait for object exit.
Module(s) involved in panic: ....."
"####..." denotes the impacted vSAN object UUID.

Environment

Backtrace in vmkernel logs may show:

2025-03-26T14:00:30.478Z cpu90:2099718)Backtrace for current CPU #90, worldID=2099718, fp=0x0
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581b920:[0x420010f7bbc0]PanicvPanicInt@vmkernel#nover+0x20c stack: 0xef, 0x420010f7bbc0, 0x0, 0x420000000001, 0x420010f7bbc0
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581b9d0:[0x420010f7c3bc]Panic_vPanic@vmkernel#nover+0x25 stack: 0x0, 0x420010f97cc1, 0xffffffffffffffef, 0x420000000010, 0x453d0581ba50
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581b9f0:[0x420010f97cc0]vmk_PanicWithModuleID@vmkernel#nover+0x41 stack: 0x453d0581ba50, 0x453d0581ba10, 0x0, 0x0, 0xa6
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581ba50:[0x420012f23f3a][email protected]#0.0.0.1+0x107f stack: 0xd1, 0xc5, 0xfc, 0x18, 0xd4
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581beb0:[0x420012fe1659][email protected]#0.0.0.1+0x12 stack: 0x319bbde900ec98, 0x4200127bba47, 0x0, 0x839, 0x432892bb5e40
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581bed0:[0x4200127bba46][email protected]#0.0.0.1+0x23f stack: 0x432892bb5e40, 0x0, 0x319bbde9017984, 0x432892bb5f18, 0x319bbde900e8b2
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581bfa0:[0x420010f9f7f8]vmkWorldFunc@vmkernel#nover+0x31 stack: 0x420010f9f7f4, 0x0, 0x453d0581f000, 0x453d0471f100, 0x453d0581f100
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581bfe0:[0x4200114d67b2]CpuSched_StartWorld@vmkernel#nover+0xbf stack: 0x0, 0x420010f44cf0, 0x0, 0x0, 0x0
2025-03-26T14:00:30.478Z cpu90:2099718)0x453d0581c000:[0x420010f44cef]Debug_IsInitialized@vmkernel#nover+0xc stack: 0x0, 0x0, 0x0, 0x0, 0x0
2025-03-26T14:00:30.497Z cpu90:2099718)VMware ESXi 8.0.3 [Releasebuild-24280767 x86_64]
########-####-####-####-############: Failed to wait for object exit.

Cause

In core dump file you may notice the "{tagLength =" value being higher than "255".

For example:

1391 uint8 i = 0;
....
1410 *tagLength = val.tagLength;
1411 for (; i < val.tagLength; ++i) {
1412 snapTag[i] = val.snapTag[i];
1413 }
(gdb) t 2499
[Switching to thread 2499 (LWP 2100116)]
#0 ZDOM_SnapshotTagQuery (snapId=snapId@entry=0, zdomObj=zdomObj@entry=0x45ddf8285c28, txn=<optimized out>, tagType=<optimized out>, tagLength=tagLength@entry=0x45ddb76ad18c, snapTag=0x43270f621d08 "/vmfs/volumes/vsan:#############-###########/########-####-####-####-############/#############/########/#######_OSDisk_1_######## (##########)/vm"...) at bora/vsan/zdom/libzdom/zdomSnapshot.c:1411
1411 for (; i < val.tagLength; ++i) {
(gdb) p key
$20 = {snapId = 0, tagType = 0 '\000', tagId = 0 '\000'}
(gdb) p val
$21 = {tagLength = 266, snapTag = "/vmfs/volumes/vsan:

In the above example, the "tagLength" has a value of "266", but the iterator 'i' for type "uint8" can only accommodate a value of up to "255".
This is the character length of the full path of the vmdk.

You may see an increase in snaptag length in circumstances involving snapshot creation as part of scheduled backup operations.

Resolution

The workaround is to create a snaptag value with less than 255 characters (the full VM file path).

In the vCenter GUI, carry out the following steps:

  1. Right-click the affected virtual machine and select "Power off", then click "Edit Settings";
  2. Locate the corresponding virtual disk (hard disk) and click the option bar on the right and select "Remove device". Then click "OK"; *Please do not select "Remove device and data" as it may delete the corresponding vmdk.*
  3. In the Virtual Machine instance UI, click "Datastores" or directly select the datastore from the inventory:
    Datacenter > Datastore, then click the corresponding vSAN ESA datastore.
  4. Click the "Files" sub-option bar and locate the corresponding vmdk according to the path.
  5. Select the vmdk and click "MOVE TO" in the option bar to select a directory with a shorter path and click the "OK" button.
  6. Return to the virtual machine instance and click "Edit Settings",
  7. Click the top right button, "ADD NEW DEVICE" and select "Existing Hard Disk",
  8. Locate the corresponding moved vmdk and click the "OK" button
  9. Power on the virtual machine.