High VM Disk Latency with Apparent Poor Network Performance

Products

• VMware vSphere ESXi

Issue/Introduction

• High disk latency on VMs on a particular host
• Performance graphs indicate the latency issue
• Host is processing all packets
• There is a high number of retransmit and SACK requests listed in the private nic statistics in the output of nicinfo.sh
• The number of packet drops is consistent with the amount of broadcast traffic not destined for the host.

Cause

• High number of SACK requests indicate large numbers of corrupt packets arriving on the interface. (Packets are corrupt ON arrival)
• This indicates an issue in the physical

Resolution

• Investigate the physical aspects of the connection:
  • Transceivers
  • Wires
  • Switches
  • Nics
    
    
• The vendors of the hardware may need to be engaged to further investigate.

Additional Information

In some instances, a reboot can clear a physical issue, that may have been caused by environmental factors such as static electricity field build up, or other similar issues.