VPXD Service Failure/crashes During Virtual Machine Disk Consolidation Due to File Lock Exceptions on vCenter Server 8.0 Update 2

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Recurring crashes of the VPXD service and VM consolidation or backup failures.
  • Analysis of service crash backtraces show entries similar to the following:

    #8  0x00005650a86c24f1 in VmMo::ConsolidateDisksStateLockedInt (this=this@entry=0x7f1944d6da90) at bora/vpx/vpxd/vm/snapshot.cpp:1484
    #9  0x00005650a86c2e2c in VmMo::ConsolidateDisks (this=0x7f1944d6da90, result=...) at bora/vpx/vpxd/vm/snapshot.cpp:1581

The above entries indicate vpxd is crashing when trying to fetch the file lock owners function:

• Disk consolidation or backup failures due to files being locked.
  • The following backtrace found in these cases shows this:

    (VPXD Backtrace):
    backtrace[08]: vpxd[Vmomi::MethodFault::Throw]   (Physical frame)
    [...]/bora/vim/lib/public/vmomi/MethodFault.h:170
    backtrace[09]: vpxd[VpxLRO::TrackAndWait]   (Physical frame)
    [...]/self/cwd/bora/vpx/common/vpxLro.cpp:1824
    backtrace[10]: vpxd[VpxLRO::TrackAndWait]   (Physical frame)
    [...]/self/cwd/bora/vpx/common/vpxLro.cpp:1724
    backtrace[11]: vpxd[VmMo::ConsolidateDisksStateLockedInt]   (Physical frame)
    [...]/cwd/bora/vpx/vpxd/vm/snapshot.cpp:1474
    backtrace[12]: vpxd[VmMo::ConsolidateDisks]   (Physical frame)
    [...]/cwd/bora/vpx/vpxd/vm/snapshot.cpp:1583

Environment

VMware vCenter Server 8.0.2

Cause

• The crash is linked to a new feature implemented in vCenter Server 8.0 Update 2 to show file lock owner information in the UI, and failure messages.
• This issue only happens when there is more than one virtual disk with file lock contention on the host.

Resolution

• Upgrade to vCenter Server 8.0 Update 2b or a later release VMware vCenter Server 8.0 Update 2b Release Notes.
• There is no workaround available for this issue on earlier 8.0 U2 builds.
• If symptoms persist after applying the vCenter Server 8.0 Update 2b patch, Please reach out to Broadcom Technical support by opening a support request, following the article Creating and managing Broadcom support cases.

 

Additional Information

• VMware vCenter Server 8.0 Update 2b Release Notes.