Error "Invalid configuration for device 'XX': triggers failure of vMotion/Cloning of for large VMs

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• Shared nothing vMotion (migration of both compute and storage) or cloning fails for very large VMs if the migration takes more than 24 hours to complete.

• Log entries similar to the following confirm the symptoms and the failure
  • Destination vCenter Server logs (var/log/vmware/vpxd/vpxd.*log)
    YYYY-MM-DDTHH:MM:SS.MMMZ error vpxd[####] [Originator@6876 sub=VmProv opID=#######] Exception while executing action vpx.vmprov.CreateDestinationVm: N3Vim5Fault17InvalidDeviceSpec9ExceptionE(Fault cause: vim.fault.InvalidDeviceSpec
  • Destination ESXi Server host management agent logs (var/run/log/hostd.log):
    YYYY-MM-DDTHH:MM:SS.MMMZ Er(163) Hostd####]: [Originator@6876 sub=######.vmx opID=######## sid=####user=vpxuser:<no user>] Device spec doesn't match up with dvport/dvpg configuration
  • Source  vCenter Server logs (var/log/vmware/vpxd/vpxd.*log)
    YYYY-MM-DDTHH:MM:SS.836Z error vpxd[####] [Originator@6876 sub=VmProv opID=#######] Get exception while executing action vpx.vmprov.CreateDestinationVm:

            --> (vim.fault.InvalidDeviceSpec) {
            -->    property = "virtualDeviceSpec.device.backing",
            -->    deviceIndex = 118,
            -->    msg = "Invalid configuration for device 'XX'.",

 

Cause

• Failure of such migrations is likely due to a missing port on the destination ESXi host.
• For any migration, vCenter Server reserves a DVS port for 24 hours.
• After 24 hours, the port reservation expires and the port is deleted by the DvsMonitor, hence the VM migrated to the destination host has no port to connect to and the migration/clone fails.

Resolution

This is default and expected behavior.

To workaround this behavior, the port reservation timeout needs to be extended to allow the migration to complete:

1. SSH to the vCenter Server appliance
2. Make a copy/backup of file /etc/vmware-vpx/vpxd.cfg:

cp /etc/vmware-vpx/vpxd.cfg /etc/vmware-vpx/vpxd.cfg.bak

     3. Add the following lines under the tab <vpxd>:

  <dvs>
    <PortReserveTimeoutInMin>7200</PortReserveTimeoutInMin>
  </dvs>

     4. Restart the vpxd service: service-control --restart vmware-vpxd

     5. Re-try the vMotion/Clone operation on the affected VMs.

     6. Once the VMs have been migrated, revert the vpxd.cfg file back to its original form:

cp /etc/vmware-vpx/vpxd.cfg.bak /etc/vmware-vpx/vpxd.cfg

     7. Restart the vpxd service: service-control --restart vmware-vpxd

Note: Value 7200  will extend port reservation timeout to 5 days (24*60*5=7200), and it should be enough to cover any time-demanding (slow/large VM (50TB+)) share-nothing vMotion/Clone.