Image level backup failed for all VMs reside on ESXi hosts using Commvault backup

Products

• VMware vSphere ESXi

Issue/Introduction

While using the Commvault backup there is a failure to connect to the host 

Error Code: [91:449]
Description: Error opening the disks for virtual machine [esxi_host_name]. Verify that the correct transport mode is selected. The operation has timed out

Environment

ESXi 8.0 installed 

There maybe a second vmk defined for the backup network

Cause

During the troubleshooting it was discovered that the backups worked over the management vmk but failed over the backup vmk.

This looks like a network issue

Resolution

Once the cause was discovered the customer was able to change the backup port to the management vmk and backups continues.

There may have been a change in Commvault for the protocols and ports being used.

Additional Information

Commvault logs show

11784 362c 06/09 22:20:42 8065207 Cn x OpenTCP Socket: Couldn't set early TCP timeout connecting to server server_name:902:19

11784 362c 06/09 22:22:28 8065207 SSL: syscall error 10054: An existing connection was forcibly closed by the remote host

11784 362c 06/09 22:22:28 8065207 SSL: connect failed (5)

11784 362c 06/09 22:22:28 8065207 CnxAuthdConnect: Returning false because SSL_ConnectAndVerify failed

11784 362c 06/09 22:22:28 8065207 CnxConnectAuthd: Returning false because CnxAuthdConnect failed

11784 362c 06/09 22:22:28 8065207 Cnx_Connect: Returning false because CnxConnectAuthd failed

11784 362c 06/09 22:22:28 8065207 Cnx_Connect: Error message: SSL failed to connect to peer

11784 362c 06/09 22:22:28 8065207 [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect: SSL failed to connect to peer

11784 362c 06/09 22:22:28 8065207 [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect to peer. Error: SSL failed to connect to peer

11784 362c 06/09 22:22:28 8065207 [NFC ERROR] NfcEstablishAuthCnxToServer: Failed to create new AuthD connection: SSL failed to connect to peer

11784 362c 06/09 22:22:28 8065207 [NFC ERROR] Nfc_BindAndEstablishAuthdCnx3: Failed to create a connection with server server_name: SSL failed to connect to peer

11784 362c 06/09 22:22:28 8065207 NBD_ClientOpen: Couldn't connect to server_name:902 SSL failed to connect to peer