Duplicate vSAN IP address cause host to become Network partitioned
search cancel

Duplicate vSAN IP address cause host to become Network partitioned

book

Article ID: 376980

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

  • vSAN host frequently becomes Network partitioned, then after sometime the host joins back without any intervention. 
  • /var/run/log/vsansystem.logs will report the change in membership count:

YYYY-MM-DDTHH:MM:SS.###Z cpux:2098825)CMMDS: CMMDSUtil_PrintArenaEntry:100: ########-####-####-####-##########: [60579888]:Adding a new Membership entry (########-####-####-####-##########) with 5 members:
YYYY-MM-DDTHH:MM:SS.###Z cpux:2098825)CMMDS: CMMDSUtil_PrintArenaEntry:100: ########-####-####-####-##########: [60644183]:Adding a new Membership entry (########-####-####-####-##########) with 1 members: 
YYYY-MM-DDTHH:MM:SS.###Z cpux:2098825)CMMDS: CMMDSUtil_PrintArenaEntry:100: ########-####-####-####-##########: [60697647]:Adding a new Membership entry (########-####-####-####-##########) with 5 members:

  • /var/run/log/vmkernel.log reports that another device in the network have the same IP address of the VMkernel adaptor that is tagged for the vSAN traffic. It will also provide the MAC address of the other device.

YYYY-MM-DDTHH:MM:SS.###Z cpuxx:2098609)Tcpip_Vmk: 124: arp: aa:bb:cc:dd:ee:ff is using my IP address ###.###.##.##  on vmkX!
YYYY-MM-DDTHH:MM:SS.###Z cpuxx:2098609)Tcpip_Vmk: 124: arp: aa:bb:cc:dd:ee:ff is using my IP address ###.###.##.##  on vmkX!
YYYY-MM-DDTHH:MM:SS.###Z cpuxx:2098609)Tcpip_Vmk: 124: arp: aa:bb:cc:dd:ee:ff is using my IP address ###.###.##.##  on vmkX!
YYYY-MM-DDTHH:MM:SS.###Z cpuxx:2098609)Tcpip_Vmk: 124: arp: aa:bb:cc:dd:ee:ff is using my IP address ###.###.##.##  on vmkX!
YYYY-MM-DDTHH:MM:SS.###Z cpuxx:2098609)Tcpip_Vmk: 124: arp: aa:bb:cc:dd:ee:ff is using my IP address ###.###.##.## on vmkX!

  • /var/run/log/vobd.log reported an event for duplicate IP address for vmk1 on ESXI host krdatvmp01. 

YYYY-MM-DDTHH:MM:SS.###Z In(14) vobd[2097620]:  [netCorrelator] 1127337923476us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for ###.###.##.## on interface vmkx. The current owner is 60:e8:5b:b8:97:4e.
YYYY-MM-DDTHH:MM:SS.###Z In(14) vobd[2097620]:  [netCorrelator] 1127336343261us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for ###.###.##.## on interface vmkx, current owner being 60:e8:5b:b8:97:4e.

  • VMkernel adaptor used for vSAN traffic can be identified using the below command.

# esxcli vsan network list

Interface:
   VmkNic Name: vmkX
   IP Protocol: IP
   Interface UUID: ########-####-####-####-##########
   Agent Group Multicast Address: ###.#.#.#
   Agent Group IPv6 Multicast Address: ####::#:#:#
   Agent Group Multicast Port: 23451
   Master Group Multicast Address: ###.#.#.#
   Master Group IPv6 Multicast Address: ####::#:#:#
   Master Group Multicast Port: 12345
   Host Unicast Channel Bound Port: 12321
   Data-in-Transit Encryption Key Exchange Port: 0
   Multicast TTL: 5
   Traffic Type: vsan

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

This issue occurs if any other device in the network have the same IP address of the VMkernel adaptor that is tagged for the vSAN traffic.

 

Resolution

Identify the other device in the network with the same IP address of the vSAN VMkernel adaptor. Change its IP address to avoid the conflict.

Additional Information

Powered by Wolken Software