Bridge instance stays in a DOWN state after Edge failover

Products

• VMware NSX

Issue/Introduction

After a Bridge instance failover, the Bridge state continues to be in a DOWN state instead of changing to Standby.

Environment

VMware NSX

Cause

Bridge port interfaces are in a DOWN state. This will cause the Bridge HA status to stay DOWN as it is not deemed healthy.

 

Interface configuration:

Bridge Port "device" can be located under the /edge/nsx-agent-state file in an Edge NSX support bundle:

            "vlan_trunks": "1-4094",
            "name": "",
            "op_state_up": true,
            "ports": [
                {
                    "bridge_port_id": 1, <<<<<<<
                    "op_state_up": false,
                   "ifuid": xxx,
                    "admin_up": true,
                   "lswitch": "xxxxx-xxxx-xxxx-xxxx-xxxxx",
                   "uuid": "xxxxx-xxxx-xxxx-xxxx-xxxxx",
                   "peer": "xxxxx-xxxx-xxxx-xxxx-xxxxx",
                    "op_state": 1,
                    "op_state_mask": 3
                }
            ],
            "admin_up": true,
            "device-admin-state": "Up",
           "uuid": "xxxxx-xxxx-xxxx-xxxx-xxxxx",
           "transport_zone_id": "xxxxx-xxxx-xxxx-xxxx-xxxxx",
            "flags": 64,
            "is_punt_port_switch": false,
            "ha_op_up": false,
            "device-state": "Up",
           "device": "edge01-lag01

If LAG is used, identify the slaves that make up the LAG group under /edge/nsx-agent-state file in an Edge NSX support bundle:

           "name": "edge01-lag01",
            "mode": "lacp-active",
            "slaves": [
                "fp-eth1", <<<<<<<
                "fp-eth3" <<<<<<<
            ],

Log information for interface state changes in /var/log/syslog:

Interfaces change from UP to DOWN:

xxxx-xxx-xxT13:16:43.917Z edge01.domain.local NSX 5311 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="phys-port" tname="dp-ipc43" level="WARN" eventId="vmwNSXPhysicalNicStatus"] {"event_state":0,"event_external_reason":"Physical port link down","event_src_comp_id":"xxxx-xxxx-xxxx-xxxx-xxxx","event_sources":{"interface_name":"fp-eth1"}}
xxxx-xxx-xxT13:16:43.917Z edge01.domain.local NSX 5311 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="phys-port" tname="dp-ipc43" level="WARN" eventId="vmwNSXPhysicalNicStatus"] {"event_state":0,"event_external_reason":"Physical port link down","event_src_comp_id":"xxxx-xxxx-xxxx-xxxx-xxxx","event_sources":{"interface_name":"fp-eth3"}}

Bridge state changes to DOWN:

xxxx-xxx-xxT13:19:20.903Z edge01.domain.local NSX 1 FABRIC [nsx@6876 comp="nsx-edge" subcomp="nsxa" s2comp="bridge" level="INFO"] bridge xxxx-xxxx-xxxx-xxxx-xxxx attached to VLAN lswitch xxxx-xxxx-xxxx-xxxx-xxxx state changed from Active to Down

Interfaces change from DOWN to UP:

xxxx-xxx-xxT14:05:14.737Z edge01.domain.local NSX 5311 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="phys-port" tname="dp-ipc43" level="INFO" eventId="vmwNSXPhysicalNicStatus"] {"event_state":1,"event_external_reason":"Physical port link up","event_src_comp_id":"xxxx-xxxx-xxxx-xxxx-xxxx","event_sources":{"interface_name":"fp-eth3"}}
xxxx-xxx-xxT14:05:14.737Z edge01.domain.local NSX 5311 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="phys-port" tname="dp-ipc43" level="INFO" eventId="vmwNSXPhysicalNicStatus"] {"event_state":1,"event_external_reason":"Physical port link up","event_src_comp_id":"xxxx-xxxx-xxxx-xxxx-xxxx","event_sources":{"interface_name":"fp-eth1"}}

Bridge state changes from DOWN to Standby:

xxxx-xxx-xxT14:08:05.085Z edge01.domain.local NSX 1 FABRIC [nsx@6876 comp="nsx-edge" subcomp="nsxa" s2comp="bridge" level="INFO"] bridge xxxx-xxxx-xxxx-xxxx-xxxx attached to VLAN lswitch xxxx-xxxx-xxxx-xxxx-xxxx state changed from Down to Standby

Resolution

The bridge HA health is determined by a combination of Edge Node health (TEP Tunnels UP) + Bridge Port health (Interfaces up).