Virtual Machine booting via Citrix Provisioning Services may fail to boot

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
VMs booting via Citrix Provisioning Services may fail to boot shortly after the Citrix bootstrap has been loaded.

VM crashes while booting "vcpu-0:This virtual machine has tried to execute an invalid part of memory" while powering on.

You will see similar entries in vmware.log:
2020-02-06T08:16:38.099Z cpu18:50421685)UserDump: 2074: vmx-vcpu-1:NWVMCDGTKC003: Dumping cartel 50782119 (from world 50421685) to file /vmfs/volumes/56a8d1c4-5455edec-922c-ecb1d77609b8/<vm name>/vmx-zdump.000
2020-02-06T08:16:39.691Z cpu18:50421685)UserDump: 2194: vmx-vcpu-1:<vm name> Userworld coredump complete.

Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5

Cause

Citrix Provisioning Services is not compatible with oversized network frames, including those produced by Network Health Check.  The condition is triggered when the Citrix Provisioning Bootstrap is connecting to the PVS Server and receives a network frame larger than 1514 bytes.  The oversized frame causes Citrix Provisioning Services Bootstrap to function incorrectly, resulting in a VM failing with the message "This virtual machine has tried to execute an invalid part of memory."

Resolution

Workaround:
Disable Network Health Check.  Eliminate other possible sources of oversized frames (greater than 1514 bytes) sent by Ethernet broadcast or by unicast to the PVS Client during boot.  Failed virtual machines can be returned to normal operation by rebooting them.