VM backup task taking a long time to complete when using a dedicated interface for "vSphere Backup NFC".

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi host has an interface configured with the tag "vSphere Backup NFC". The NBD traffic will use this interface.
  
  
• The backup interface and the backup media servers are in the same L2 network.
  
  
• The ESXi management vmk interface and the backup media servers are in different networks.
  
  
• The backup media servers are reachable via both the ESXi management vmk and the backup vmk interface.
  
  
• In the VDDK VixDiskLib.log, there are errors reported while trying to open the vmdk file:
  
  10/30 11:44:13 VixDiskLib: VixDiskLib_OpenEx: Open a disk.
10/30 11:44:13 NBD_ClientOpen: attempting to create connection to vpxa-nfcssl://[datastore_name] vm_name/vm_name.vmdk@{nfcip:XX.XX.XX.5,hostname:esxi_hostname}:902
10/30 11:44:23 CnxOpenTCPSocket: Timed out connecting to server XX.XX.XX.5:902: Operation now in progress
10/30 11:44:23 CnxAuthdConnect: Returning false because CnxAuthdConnectTCP failed
10/30 11:44:23 Cnx_Connect: Returning false because CnxConnectAuthd failed
10/30 11:44:23 Cnx_Connect: Error message: Failed to connect to server XX.XX.XX.5:902
10/30 11:44:23 [NFC ERROR]NfcNewAuthdConnectionEx: Failed to connect: Failed to connect to server XX.XX.XX.5:902
10/30 11:44:23 [NFC ERROR]NfcNewAuthdConnectionEx: Failed to connect to peer. Error: Failed to connect to server XX.XX.XX.5:902
10/30 11:44:23 DISKLIB-DSCPTR: DescriptorOpenNbd: Failed to open NBD extent 'vpxa-nfcssl://[datastore_name] vm_name/vm_name.vmdk@{nfcip:XX.XX.XX.5,hostname:esxi_hostname}:902': NBD_ERR_NETWORK_CONNECT
10/30 11:44:23 DISKLIB-LINK  : "vpxa-nfcssl://[datastore_name] vm_name/vm_name.vmdk@{nfcip:XX.XX.XX.5,hostname:esxi_hostname}:902" : failed to open (NBD_ERR_NETWORK_CONNECT).  
10/30 11:44:23 DISKLIB-CHAIN : DiskChainOpen: "vpxa-nfcssl://[datastore_name] vm_name/vm_name.vmdk@{nfcip:XX.XX.XX.5,hostname:esxi_hostname}:902": failed to open: NBD_ERR_NETWORK_CONNECT.
10/30 11:44:23 DISKLIB-LIB   : Failed to open 'vpxa-nfcssl://[datastore_name] vm_name/vm_name.vmdk@{nfcip:XX.XX.XX.5,hostname:esxi_hostname}:902' with flags 0xe NBD_ERR_NETWORK_CONNECT (2338).
10/30 11:44:23 VixDiskLib: Unable to locate appropriate transport mode to open disk. Error 13 (You do not have access rights to this file) at 6849.
10/30 11:44:23 VixDiskLib: VixDiskLib_OpenEx: Cannot open disk [datastore_name] vm_name/vm_name.vmdk. Error 13 (You do not have access rights to this file) at 7426.
10/30 11:44:23 VixDiskLib: VixDiskLib_Open: Cannot open disk [datastore_name] vm_name/vm_name.vmdk. Error 13 (You do not have access rights to this file) at 7504.

Environment

VMware vSphere ESXi

Cause

TCP port 443 is blocked between the ESXi management vmk network and the backup media server network.

Though there is a dedicated vmk interface for backup, it will be used only for NBD traffic over port 902. The opening of disks will still need TCP port 443 to be open between the ESXi management vmk network and the backup media server network.

Resolution

Open TCP port 443 between the ESXi management vmk network and the backup media server network.