ESXi hosts disconnecting after changing IP address of vCenter

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• Managed host disconnects from VMware vCenter Server.
• Managed hosts enter Not Responding mode for 90 seconds repeatedly.
• vCenter Server does not receive heartbeats from the ESXi hosts.
• VirtualCenter.AutoManagedIPV4 points to an incorrect VC IP address.
• The new IP address of the vCenter is not properly updated in the database of the vCenter. To verify this connect to the VCDB and run the following query:
• Connect to the vCenter Server database

  /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

• Check the IP addresses that vCenter has in the DB for itself and what the hosts have

  select * from vpx_parameter where name like '%Managed%';
  select id, dns_name, management_ip, local_ip_address from vpx_host;

Environment

vCenter 7.x

vCenter 8.x

Cause

The issue occurs since the newly updated ip address of the vCenter which was edited via Management Interface ( VAMI ) was not in the vCenter database. 

Resolution

1. Snapshot and/or backup the vCenter according to best practices
2. Stop the vpxd service
   

   service-control --stop vmware-vpxd

3. Connect to the database
   

   /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

4. Run the below queries to update the old IP with the new, desired IP (where #.#.#.# is the new IP address of vCenter)
   

   update vpx_parameter set value = '#.#.#.#' where name = 'VirtualCenter.AutoManagedIPV4';
   update vpx_parameter set value = '#.#.#.#' where name = 'VirtualCenter.ManagedIP';
   update vpx_host set management_ip = '#.#.#.#' where id = '0';
   update vpx_host set local_ip_address = '#.#.#.#' where id = '0';

5. Exit the database with "\q" and start the vpxd service
   

   service-control --start vmware-vpxd

6. Disconnect and reconnect the affected hosts.