Loss of network connectivity when Cisco port security is configured on the physical switch

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

If you are using a Cisco 6500 Switch and have port security configured on the physical switch, you may experience these symptoms:

• After vMotion, virtual machine loses network connectivity
• When teaming network adapters and failing one of them, a virtual machine or the ESXi host loses network connectivity
• A limited number of TCP/IP connections can be established
• A virtual machine cannot ping any other host on the physical network
• A virtual machine cannot ping the gateway IP address
• If a virtual machine is restarted, it loses network connectivity until the NIC is disabled and re-enabled

Notes:

• Virtual machines can ping each other on same ESXi network
• The virtual machine can ping its own IP address if the virtual NIC is configured with a static IP address
• When the network connection is disabled and enabled inside the virtual machine, the network connection is restored, and the virtual machine can ping other machines on the network

Environment

VMware vSphere ESXi 6.7

VMware vSphere ESXi 7.0

VMware vSphere ESXi 8.0
VMware vCenter Server 6.7

VMware vCenter Server 7.0

VMware vCenter Server 8.0

Resolution

Cisco Port security restricts the input to an interface by limiting and identifying MAC addresses of the virtual machines that are allowed to access the port. When a secure MAC addresses is assigned to a secure port, the port does not forward packets with source addresses outside the group of defined addresses.

 

If port security is enabled on the switch, the command show mac-address-table shows the virtual network adapters as having static MAC entries. When the virtual machine proceeds to connect through a different port (for example, after vMotion or a network adapter failover), its traffic is blocked on the new port. Network connection issues may occur if a switch port does not allow traffic from multiple MAC addresses.

For more information, see Configuring Port Security in the Cisco Catalyst 6500 Release 12.2SXH and Later Software Configuration Guide.

 

There are a few ways to resolve this issue:

• Disable port security.
• Configure port security with proper port numbers. This option provides some security.
• Configure a secure static MAC address. This is the most secure option.

Disabling port security

Caution: This option does not provide any security.

 

To disable port security on the Cisco switch interface, run this command at Cisco switch port:

no switchport port-security

Configuring port security with proper port numbers

Run this command at Cisco switch port to set a maximum number of secure MAC addresses for the interface:

Switch(config-if) # switchport port-security maximum value

where value is the maximum number of MAC addresses

Note: The default maximum value is 1. Enter a value from 1 to 1024. Ensure that you enter a maximum value that allows for the number of virtual network adapters on the ESX host.

Configuring a secure static MAC address

 

To configure a secure static MAC address, run this command at Cisco switch port:

Router(config-if)# switchport port-security mac-address [sticky] mac_address [vlan vlan_ID]

where mac_address is the MAC address that you want to configure as static and vlan_ID is the VLAN in which the MAC address resides

To delete a static MAC address:

1. Run the command:
   
   Router(config-if)# no switchport port-security mac-address [sticky] mac_address
   
   wheremac_addressis the MAC address that you want to delete
    
2. After removing the offending MAC address the switch port link goes down. Run this command to enable the switch port:
   
   Switch(config-if) # no shut