SFP Port Auto-Negotiation Limitations on VMware SD-WAN Edge 6x0 Models

Products

• VMware SD-WAN by VeloCloud

Issue/Introduction

Symptoms:

When using a VMware SD-WAN Edge 6x0 series (610, 610-LTE, 610N, 620, 620N, 640, 640N, 680, 680N), a fiber-based SFP link does not come up when connecting to a peer which has auto-negotiation set.

Environment

VMware SD-WAN by VeloCloud

Cause

Edge 6x0 models are designed with Intel Ethernet controller on the SFP ports that do not support speed or duplex auto-negotiation.  This is an industry-wide firmware limitation with the Intel x552/x553 controllers for Fiber SFPs and there is no driver level resolution that can be implemented to correct this limitation.

These limitations can cause interoperability issues when the peer-end Fiber SFP is driven by a NIC chip with auto-negotiation enabled. We have found this to be device dependent and results can be unpredictable, even if both ends of the link are using single-speed SFPs.

Resolution

This is an industry-wide firmware limitation with the Intel x552/x553 controllers for Fiber SFPs and there is no driver-level resolution that can be implemented to correct this limitation. The only way to remediate this issue is to use one of the workarounds listed below.

Workaround:

As a result of these limitations with the Edge 6x0's Intel x553 NIC chip and its firmware, when using a fiber-based SFP on an Edge 6x0, the customer should always turn off auto-negotiation on the peer side and set the peer SFP at a fixed speed to match the speed of the SFP port on the Edge.

If the customer cannot control the peer settings, we suggest using an intermediate device such as a switch or media converter as described below.

To summarize, these are the workaround options when using a fiber SFP in an Edge 6x0 series:

• Option 1: Disable auto-negotiation on the peer-end and set the speed to match the SFP that is inserted into the Edge 6x0. (Recommended)

• Option 2: Use a small managed switch between the Edge and peer. Then on the switch port connecting to the Edge, disable auto-negotiation and set the speed to match the SFP that is inserted into the Edge 6x0.

• Option 3: Use a copper SFP in the Edge 6x0 and connect to a media converter (for example, a TP-LINK MC220L) to convert from the copper link to the peer fiber link. Copper SFPs do not have an issue with auto-negotiation on Edge 6x0's and thus this option removes that setting from the equation as long as a media converter is also used.