vMotion is not working within cluster between ESXI host.

Products

• VMware vSphere ESX 6.x
• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

• vMotion Failed at 20 %
• vMotion is not working within cluster between ESXi host.
• Error while doing vMotion :

• hostd.log :

  [YYYY-MM-DDTHH:MM:SS] verbose hostd[2332791] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/[datastore]/[vm_directory]/vm_name.vmx opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] VMotionPrepare: MigrateFromDest message: The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct.
  -->
  [YYYY-MM-DDTHH:MM:SS] verbose hostd[2100796] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[datastore]/[vm_directory]/vm_name.vmx opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] VMotionStatusCb [5302968543718998800]: Failed with error [N3Vim5Fault20GenericVmConfigFaultE:0x000000cac895a890]
  [YYYY-MM-DDTHH:MM:SS] info hostd[2100796] [Originator@6876 sub=Libs opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] VigorTransportClientManageConnection: connection closed
  [YYYY-MM-DDTHH:MM:SS] verbose hostd[2100796] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[datastore]/[vm_directory]/vm_name.vmx opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] VMotionStatusCb: Firing ResolveCb
  [YYYY-MM-DDTHH:MM:SS] info hostd[2100796] [Originator@6876 sub=Vcsvc.VMotionDst.5302968543718998800 opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] ResolveCb: VMX reports needsUnregister = true for migrateType MIGRATE_TYPE_VMOTION
  [YYYY-MM-DDTHH:MM:SS] info hostd[2100796] [Originator@6876 sub=Vcsvc.VMotionDst.5302968543718998800 opID=m3wuhb1q-707027-auto-f5jp-h5:70044725-16-01-32-afca] ResolveCb: Failed with fault: (vim.fault.GenericVmConfigFault) {
  -->    faultMessage = (vmodl.LocalizableMessage) [
  -->       (vmodl.LocalizableMessage) {
  -->          key = "msg.checkpoint.migration.nodata",
  -->          message = "The vMotion failed because the destination host did not receive data from the source host on the vMotion network.  Please check your vMotion network settings and physical network configuration and ensure they are correct.",
  -->       }
  -->    ],
  -->    reason = "The vMotion failed because the destination host did not receive data from the source host on the vMotion network.  Please check your vMotion network settings and physical network configuration and ensure they are correct.",
  -->    msg = "The vMotion failed because the destination host did not receive data from the source host on the vMotion network.  Please check your vMotion network settings and physical network configuration and ensure they are correct.
  --> "
  --> }

  
  Log File Name: /var/run/log/vobd.log

  [YYYY-MM-DDTHH:MM:SS] [netCorrelator] 3448164565985us: [esx.problem.net.vmknic.ip.duplicate]Duplicate IP address detected for xxx.xxx.xxx.xxx on interface vmk1, current owner being 00:xx:56:xx:da:b8.
  [YYYY-MM-DDTHH:MM:SS] [netCorrelator] 3465283783726us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for xxx.xxx.xxx.xxx on interface vmk1. The current owner is 00:xx:56:xx:da:b8.
  [YYYY-MM-DDTHH:MM:SS] [netCorrelator] 3465384259816us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for xxx.xxx.xxx.xxx on interface vmk1, current owner being 00:xx:56:xx:da:b8.
  [YYYY-MM-DDTHH:MM:SS] [netCorrelator] 3466432293534us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for xxx.xxx.xxx.xxx on interface vmk1. The current owner is 00:xx:56:xx:da:b8.
  [YYYY-MM-DDTHH:MM:SS] [netCorrelator] 3466532801440us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for xxx.xxx.xxx.xxx on interface vmk1, current owner being 00:xx:56:xx:da:b8.

• Same IP assigned to vmotion Network having different MAC :

 

Environment

VMware vSphere ESXi 8.0

VMware vSphere ESXi 7.0

VMware vSphere ESXi 6.0

Cause

• This type of issue or error traces can occur only when we used similar IP for vMotion Network which was already assigned to any other device or interface.
• IP will conflict and due which migration failed.

Resolution

• As per the mentioned details and the traces, Its a Network issue which needs to check with internal Physical Network Team.
• Need to change the IP of vMotion Network or other device where same IP is assigned ( Need to assigned new Blank IP or which is not in used ).

Additional Information

For troubleshooting vMotion Network issue :

https://knowledge.broadcom.com/external/article/321012/troubleshooting-vmotion-network-connecti.html