VPXD service crashes while connecting host to the vCenter server

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• Unable to connect a host which was in a "not responding" state.
• Connecting the host results in vpxd service crash on vCenter,
• vpxd crash can be detected from the /var/log/vmware/vpxd/vpxd.log on vcenter

YYYY-MM-DDT##:##:##:###Z error vpxd[#######] [Originator@#### sub=Default opID=XXXXXX-#######-auto-#XXXX-h5:########-##] [Vdb :: VdbField] Invalid value written to column UUID in table XXXX
YYYY-MM-DDT##:##:##:###Z error vpxd[#######] [Originator@#### sub=Default opID=XXXXXX-#######-auto-#XXXX-h5:########-##] [Vdb :: VdbField] String too large: 160 > max(128)
YYYY-MM-DDT##:##:##:###Z error vpxd[#######] [Originator@#### sub=InvtListener opID=XXXXXX-#######-auto-#XXXX-h5:########-##] entity remove listener threw: N5Vmomi5Fault11SystemError9ExceptionE(Fault cause: vmodl. fault.SystemError
-- > )
-- > [context]zKq7AVECAQAAAH1IQAEhdnB4ZAAA7tg3bGlidm1hY29yZS5zbwAAp3gsACVsLQAL6TIBqbdvdnB4ZAABbblvgb9UcQGB8V5xAYFwLLUBARCWjwGfCokBzE6TAfhkdAGavIgBm/WMAeX4 jAFgM40Bbq0NAXKwjQGD0nMB7RV0gou6BQFsaWJ2aW0tdHlwZXMuc28AgRi9YQGBE8BgAYFUwWABgYLQYAGB5f1fAYG
p2ABAKdJIwBPnyMArGQ3A4d/AGxpYnB0aHJlYWQuc28uMAAELzYPbGliYy5zby42AA == [/context]
YYYY-MM-DDT##:##:##:###Z panic vpxd[######] [Originator@#### sub=Default opID=XXXXXX-#######-auto-#XXXX-h5:########-##]

-- > Panic: NOT_REACHED bora/vpx/vpxd/invtListener.cpp:195

-- > Backtrace:
-- > [backtrace begin] product: VMware VirtualCenter, version: 7.0.3, build: build-20990077, tag: vpxd, cpu: x86_64, os: linux, buildType: release
-- > backtrace[00] libvmacore.so[0x0037DA77]
-> backtrace[01] libvmacore.so[0x002C78D3]: Vmacore :: System :: Stacktrace :: CaptureFullWork(unsigned int)
-- > backtrace[02] libvmacore.so[0x002D6B69]: Vmacore :: System :: SystemFactory :: CreateBacktrace(Vmacore :: Ref<Vmacore :: System :: Backtrace>&)
-- > backtrace[03] libvmacore.so[0x00370BC3]
-- > backtrace[041 libvmacore.sol0x00370CDFl: Vmacore :: PanicExit(char const*)

Environment

VMware vCenter Server 7.x and 8.x

VMware vSphere ESXi 7.x and 8.x

Cause

This issue is due to an excessively long datastore URL which is mounted on the host, exceeding the maximum limit of 128 bytes, or 64 characters.

Resolution

Currently there is no resolution. The following workarounds can be implemented.

Workaround 1:

• Unmount the datastore which has exceeded the datastore URL limit from the host.
• If the unmount option is grayed out, delete the datastore from the host client.

NOTE: Ensure there is no reference to the datastore by any existing VMs, log files or snapshots.

• Check Error: "Device or Resource Busy" while attempting to delete a Datastore. for further details.

Workaround 2: 

• Re-signature the UUID of the volume from ESXi SSH using the command:

esxcli storage vmfs snapshot resignature -l label | -u uuid