multiple ESXi hosts but not all get NO_RESPONSE and soon CONNECTED at the same time

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Multible hosts but not all get NO_RESPONSE at the same time
• All issue hosts get CONNECTED back in several seconds
• On vpxd.log it shows connection handshake timed out 
  vpxd[01058] [Originator@6876 sub=HttpConnectionPool-000097 opID=PollQuickStatsLoop-xxxx] Failed to get pooled connection; <cs p:00007f97440e4ef0, TCP:<host FQDN>:443>, SSL(<io_obj p:0x00007f9710ecd7d8, h:162, <TCP '<vCenter IP> : 59736'>, <TCP '<host IP> : 443'>>), duration: 22695msec, N7Vmacore3Ssl12SSLExceptionE(SSL Exception: error:140000DB:SSL routines:SSL routines:short read: The connection was closed by the remote end during handshake.)
• On rhttpproxy.log on issue host, it also shows Handshake Timeout.
  error rhttpproxy[19247990] [Originator@6876 sub=Default] SSL Async Handshake Timeout : Read timeout after approximately 25000ms. Closing stream <SSL(<io_obj p:0x000000a3d916a9b8, h:17, <TCP '<Host Ip> : 443'>, <TCP '<vCenter IP> : 59736'>>)>

Environment

ESXi 

Cause

There is some network issue during handshake between vCenter and ESXi.

Resolution

To identify the cause, Please capture network package on vCenter and ESXi ( Collecting network packets using the lightweight PacketCapture) if the issue happens frequently and Create a Support Case Broadcom Support..