VMware vCenter Server IP address change/incorrect causes VMware ESXi hosts to disconnect

Products

VMware vCenter Server

Issue/Introduction

After changing the IP Address of the vCenter Server, all ESXi hosts are showing as disconnected within the vCenter's inventory
When manually reconnecting an ESXi host, it is disconnected again after a short time
Getting following error message in hosts in the cluster in the vCenter: Agent can't send heartbeats because socket address structure initialization is failing
vpxa.log: YYYY-MM-DDThh:mm:ssZ Er(163) Vpxa[2099707]: [Originator@6876 sub=Heartbeat opID=vpxaHeartbeat.cpp:218-117a92xx] getaddrinfo of server address failed; ' xxx.xx.x.xx', r: -2(Name or service not known)

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x

Resolution

Note: Before starting, see Restarting the Management agents on an ESX Server for important information on restarting the mgmt-vmware service.

The ESXi hosts disconnect because they store the IP address of the vCenter Server in configuration files on each of the individual servers. This incorrect address continues to be used for heartbeat packets to vCenter Server.

There are two methods to get the ESXi hosts connected again. Try each one in the order specified, testing the results each time.

Before performing either method, ensure that the Managed IP Address of the vCenter Server is updated to reflect the new IP address being used. Also ensure that there are not any additional trailing whitespaces in the IP address. For more information, see Verifying the vCenter Server Managed IP Address.

Method 1 - edit the vpxa configuration

Up until ESXi 7.0 Update 2e - edit the vpxa.cfg file on ESXi:
1. Log in as root to the ESXi host with an SSH client.
2. Open the vpxa.cfg file using a text editor and change the serverIp parameter to the new IP of the vCenter Server.
  1. In ESXi 5.x up until 7.0 U2e the vpxa.cfg can be found in /etc/vmware/vpxa/vpxa.cfg.
3. Save the changes and exit.
4. Restart the management agents on the ESXi. For more information, see Restarting the Management agents on an ESX Server.
5. Restart the vCenter Server service. For more information, see Stopping, starting, or restarting the vCenter Server service.
In ESXi 7.0 Update 3 and later - edit the config store:
- In ESXi 7.0 Update 3, the vpxa.cfg file no longer exists, as the vpxa configuration was moved to the ESXi config store. Therefore, in the event that /etc/vmware/vpxa/vpxa.cfg cannot be found on ESXi 7.0, follow these steps instead:

1. 1. Export the current vpxa configuration from the config store:
    configstorecli config current get -c esx -g services -k vpxa_solution_user_config -outfile vpxa.json
  2. Create a backup of the file containing the current configuration:
    rm -f vpxa.json.bak; cp vpxa.json vpxa.json.bak -f
  3. Replace the old vCenter IP address with the new one (replace <vcenter_ip> with the correct IP address):
    sed -e 's/ "server_ip".*/ "server_ip": "<vcenter_ip>",/g' -i vpxa.json
  4. Import the changed configuration into the ESXi config store:
    configstorecli config current update -c esx -g services -k vpxa_solution_user_config -infile vpxa.json
  5. Verify that the IP was correctly changed:
    configstorecli config current get -c esx -g services -k vpxa_solution_user_config
  6. Restart vpxa to apply the change:
    /etc/init.d/vpxa restart
    Note: If method 1 does not work, apply method 2.

Method 2 - disconnect/reconnect the ESXi in vSphere Client

Note: Do not remove the ESXi host from vCenter inventory if the host is attached to a Distributed Switch (vDS). For more information, see Moving an ESX/ESXi host with vDS from one vCenter Server to another.

From the vSphere Client, right-click the ESXi host and click Disconnect.
From the vSphere Client, right-click the ESXi host and click Reconnect. If the IP is still not correct, go to step 3.
From the vSphere Client, right-click the ESXi host and click Remove.
Caution: After removing the host from vCenter Server, all the performance data for the virtual machines and the performance data for the host is lost.
Click New > Add Host.
Enter the information used for connecting to the host.

If the ESXi keeps reconnecting continuously and "vpxa_solution_user_config" is reverting back to old VC IP:

configstorecli config current get -c esx -g services -k vpxa_solution_user_config

Step 1: Verify Runtime settings for vCenter Server

To update the vCenter Server managed address, follow these steps:

Log in to the vSphere Client and navigate to the vCenter Server instance.
Click on the Configure tab.
Under Settings, select General.
Click Edit to open the Edit vCenter Server Settings dialog box.
In the dialog box, select Runtime Settings.
Verify that the vCenter Server managed address field is reflecting the OLD IP address.
Update the vCenter Server managed address field with the NEW IP address.
Enter a unique ID in the vCenter Server unique ID field (optional).
Enter the name of the vCenter Server system in the vCenter Server name field (optional).
Click Save to save the changes.

Step 2: Restart the vCenter Server

After updating the vCenter Server managed address, restart the vCenter Server to apply the changes.

If an incorrect Managed IP Address is configured for VirtualCenter.AutoManagedIPV4 and VirtualCenter.AutoManagedIPV6, the vCenter Server does not receive heartbeats from the ESX/ESXi host causing it to go into a disconnected state.

Determine the managed IP address of the vCenter Server:

Connect to vCenter Server with the vSphere Client
Click on the vCenter Server's name
Click Configure > Advanced Settings >
Under the "Name" column, click on the filter icon to the right of "Name"
In the search box, search for VirtualCenter.AutoManagedIPV4 and VirtualCenter.AutoManagedIPV6
Verify and record the listed value under these 2 entries.

Update the "vCenter Server managed address" with the new IP address using the steps shared earlier.

Additional Information

For related information, see Verifying the VirtualCenter Managed IP Address.