vSphere HA agent on this host could not reach isolation address '#.#.#.#', reported on ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

• This article explains how this issue can occur and provides guidance on how to work around it.
  
  
  
  • In the vCenter user interface, in the Hosts and Clusters view, with the ESXi host selected, you see the message "vSphere HA agent on this host could not reach isolation address  '6.#.#.#'.
  •  6.#.#.# is configured as gateway of DefaultTcpIpStack.as below:
    
    
    
  • To check this:
    1. Connect to the vCenter Server through the vSphere Client
    2. Select Hosts and Clusters, select the ESXi host object
    3. Select Configure > Networking > TCP/IP configuration
    Note the default TCP/IP stack gateway is set to  '#.#.#.#'

Environment

VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7
VMware vSphere ESXi 7.0

Cause

• A new vSwitch and vmk port is created automatically when Integrated Management Module (IMM) is connected.
• If 0.0.0.1/24 is configured as IP address by IMM, an ESXi bug will be triggered and  '6.#.#.#' will be set as gateway of DefaultTcpIpStack,
  You will see an error because this is not a valid gateway address.

Resolution

This issue is resolved in VMware ESXi 7.0 Update 1 (build number 16850804).

Workaround:

1. Modify the IMM configuration to make sure a legal IPv4 address is set, but not 0.0.0.1/24

    For information about modifying the IMM configuration, consult your ESXi server hardware vendor or documentation.
 

2. Modify ESXi configuration to clear the notification "vsphere ha agent on this host could not reach isolation address '6.#.#.#':

• Connect to the vCenter Server through the vSphere Client
• Select Hosts and Clusters, select the ESXi host object
• Select Configure > Networking > VMkernel adapters
• Select each vmk to check if the gateway is set to  '6.#.#.#'.
• With a vmk selected, In the pane below the adapter list select the All tab
  
  If the display below under IPv4 settings shows the Default gateway is  '6.#.#.#':

3. Either select the 'X' icon to delete the vmk, or

    Select the pencil (edit) icon to edit the settings:

• Select IPv4 settings
• Select Use static IPv4 settings
• Enable Override default gateway for this adapter
• Set the gateway to a valid address 
  • If deleting the vmk:
    

 

• • If modifying the vmk gateway to a valid IP address:
    

 

4. Check the management network gateway setting in the ESXi host Direct Console User Interface (DCUI)

a. Access to the ESXi host DCUI.
    Note: This can be done by direct connection to the server hardware, back-end management system such as iDRAC, iLO, UCS, or from an SSH session per KB 311478 .
b. Select IPv4 Configuration. If the gateway is set to  '6.#.#.#' and use the Enter key to change settings
c. Set the default gateway to a valid IP address.


d. Restart management agents. Refer to Restarting the Management agents in ESXi (320280)

Additional Information

This issue can be reproduced by setting up ESXi with default network configuration, vmk0 and vSwitch0:
a. esxcli network vswitch standard portgroup add --portgroup-name="test-pg" --vswitch-name=vSwitch0
b. esxcli network vswitch standard portgroup list
c. esxcli network ip interface add --interface-name=vmk1 --portgroup-name=test-pg
d. esxcli network ip interface ipv4 set --interface-name=vmk1 --type=static --ipv4=0.0.0.1 --netmask=255.xx.xx.xx

Impact/Risks:
vSphere HA agent cannot be implemented on affected ESXi host.

Changing network settings using these commands may make the host unusable and can necessitate a reboot to recover.