Failed to clone and Could not complete network copy for file during template publishing

Products

• VMware vCenter Server
• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x
• VMware vSphere ESXi

Issue/Introduction

• Using vCenter appliances that are in Linked mode with content libraires, creating the subscription and initiating a publish for templates you have the following error on the UI: 
  
  Failed to clone <vm> on <esxi host> ....Could not complete network copy for file /vmfs/volumes/#####/<vm>.vmdk
  
  
  
  Event Type Description: Cloning a virtual machine failed
  
  
• On source host vmkernel.log:
  
   vmkwarning: cpu2:17293857)WARNING: XVMotion:.... timed out while waiting for disk 1's queue count to drop below the maximum limit of 32768 blocks. This could indicate either network or stora$
  
  

• On destination ESXi host vmkernel.log the following regarding bandwidth of copy (BW) is seen:
  
   XVMotion: 4320: .... D: Write IOPS:277, BW:8 MB/s, Lat(us)MAX:6567, MIN:304, AVG[863 765 702 769 804 731 744 802 822 802 710 772]

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

Bandwidth available during data copy is bellow minimum recommended of 250Mb/s and copy doesn't make progress to drain disk queue in 20 seconds

Resolution

Increase the available network bandwidth between the two ESXi hosts involved in the copy to at least 250Mb/s

Additional Information

vMotion bandwidth requirements
Using iPerf for Testing vSAN Network Performance