Reconnecting or adding an ESXi host to vCenter fails with the error: "Agent can't send heartbeats because socket address structure initialization is failing"

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

Symptoms:

• Reconnecting or adding an ESXi host to vCenter Server displays the error:
  Agent can't send heartbeats because socket address structure initialization is failing
  
  
• The vpxa.cfg file at /etc/vmware/vpx/vpxa.cfg on the affected host contains the fully qualified domain name (FQDN) of the vCenter Server. This can also occur if there is a trailing whitespace on a valid IP. 
  Example:
  
  <vpxa>
  <hostIp>host-ip</hostIp>
  <hostKey>xxxxxxx-xxx-xxxxxxx-xxxxxxxxxx</hostKey>
  <hostPort>443</hostPort>
  <licenseExpiryNotificationThreshold>15</licenseExpiryNotificationThreshold>
  <memoryCheckerTimeInSecs>30</memoryCheckerTimeInSecs>
  <serverIp>FQDNofvCenterServer</serverIp>
  <serverPort>902</serverPort>
  </vpxa>
  
  
• The vpxa.log file at /var/log/vpxa/log on the affected host displays an error similar to:
  YYYY-MM-DD_TIME [FFC991A0 error 'Heartbeat' opID=SWI-885deXX1] Unable to get the socket address structure for the server address :FQDNofvCenterServer

Environment

VMware vCenter Server

Cause

The issue occurs when the vpxa.cfg file is created on the ESXi host using the fully qualified domain name of the vCenter Server from the managed_ip value within the vCenter Server database. This value is expected to be an IP address.

Resolution

To resolve this issue:

• Verify that the ESXi hosts are able to resolve the fully qualified domain name of the vCenter Server to which they are connected.
• Verify that the vCenter Server Managed IP address is correct. For more information, see Verifying the VMware vCenter Server Managed IP Address.

Additional Information

NOTE: Make sure all vCenters in ELM are shut down and take a snapshot of all nodes for backup. For standalone vCenters, a powered-on snapshot will be sufficient. Please check the below article for snapshot best practices:

 

• Best practices for using VMware snapshots in the vSphere environment
• Snapshot Best practices for vCenter Server Virtual Machines
• VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice

To verify the vCenter Server Managed IP address within the vCenter Server database:

1. Connect to the vCenter Embedded vPostgres Database from the vCenter Server Appliance:

$ /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

 

2. Show the FQDN:

$ select id,dns_name,management_ip from vpx_host where dns_name like '<vCenter_FQDN>'

 

To update the Managed IP address directly within the vCenter Server database:

1. This involves database changes hence a snapshot of vCenter as well as backup of the vCenter is a must. File-Based Backup and Restore of vCenter Server

2. Stop the vCenter vpxd service:
  $ service-control --stop vmware-vpxd

3. Execute the following SQL query:
  $ update vpx_host set management_ip = <vCenter_Server_IP_Address>

4. Start the vCenter Server service.
  $ service-control --start vmware-vpxd


5. Reconnect the ESXi host.