Veem backup integration failing to install

Products

• VMware vSphere ESXi

Issue/Introduction

Backup solutions (specifically Veeam) fail to connect to ESXi hosts within a vSphere cluster.

Upon inspection of the ESXi host certificate:

The Subject or Common Name (CN) field displays the IP address of the host rather than the expected Fully Qualified Domain Name (FQDN).

Environment

VMware vSphere ESXi 8.0 U3

Cause

The hostname was set during initial deployment of ESXi with the IP address instead of an FQDN

Resolution

To resolve this issue, you must first disable the external enforcement of the hostname, and then regenerate the ESXi certificates.

NOTE: You may need to reconnect the host to vCenter during some steps in this process. Reconnect a Managed Host

Step 1: Correct the Hostname

1. Set the correct FQDN on the ESXi host:

   esxcli system hostname set --fqdn=hostname.domain.com
   

2. Reboot the host and verify the hostname persists:

   hostname -f

NOTE: If this hostname reverts, you may need to engage the hardware vendor (e.g., HP Support) or check the BIOS/iLO settings to disable hostname enforcement(e.g., Simplivity)

Step 2: Regenerate Certificates Once the hostname persists correctly across reboots, regenerate the internal certificates to match the FQDN.

Option 1 Via the vCenter:

1. 1. Navigate to HomeHosts and Clusters and select a host.
   2. Right-click the host and select Connection > Connect from the pop-up menu.

Renew or Refresh ESXi Certificates

Option 2 Via the host:

1. 1. Connect to the host via SSH
   2. Login using the root account
   3. Run the following command to generate a new self-signed certificate:

      /sbin/generate-certificates

   4. Restart the ESXi management agents:

      /etc/init.d/hostd restart && /etc/init.d/vpxa restart

      Regenerating Self-Signed Certificates on Standalone ESXi Hosts

Additional Information

For details on certificate renewal limitations when using Custom Mode (thumbprint), see: Unable to refresh/renew ESXi certificate (Broadcom KB 377969)