Connectivity and performance issues encountered when using Port Mirroring with Encapsulated Remote Mirroring (L3) Source

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

When using Port Mirroring and "Encapsulated Remote Mirroring (L3) Source" as the session type on a large number of dvPorts, you might see:

• Packet loss / poor networking performance for VMs or ESXi hosts
• ESXi hosts are intermittently disconnecting from vCenter, and you see messages similar to the example below in the host's hostd.log

  • YYYY-MM-DDT##:##:##.###Z warning hostd[#######] [Originator@6876 sub=IO.Connection opID=########] Failed to write buffer to stream; <io_obj p:0x#########, h:##, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : #####'>> e: 105(No buffer space available), async: false, duration: 0msec
    
    YYYY-MM-DDT##:##:##.###Z error hostd[#######] [Originator@6876 sub=SoapAdapter.HTTPService opID=########] Failed to write to response stream; <<io_obj p:0x0x#########, h:##, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : #####'>>, ######-######-######-######>, N7Vmacore15SystemExceptionE(No buffer space available)

Cause

Port mirroring can generate a lot of network traffic, and there may be a performance impact to the ESXi host and its VMs when a high number of dvPorts are being mirrored with the Encapsulated Remote Mirroring (L3) Source session type.

Resolution

Port mirroring is used to analyze and debug data or diagnose errors on a network, i.e. troubleshooting.

As a best practice, limit its use to a maximum of 6 VMs (per host) at a time for short periods of troubleshooting and remediation.

Additional Information

What is Port Mirroring