VMware NSX Tier 0 Service Group Failover and Edge Node Unreachable

Products

• VMware NSX

Issue/Introduction

• Intermittent BGP flapping on the Tier-0 gateway.
• T0 A/A Stateful configuration automatically changes to Stateless (Stateful toggled to False).
• Edge node logs report tier0_service_group_failover events.
• Alarms indicate peer node unreachability.

• Below error messages observed in the Edge node logs:

  • NSX 1 - [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-agent" s2comp="nsx-monitoring" entId="0000####-0000-####-0000-####0000####" tid="1" level="ERROR" eventState="On" eventFeatureName="high_availability" eventSev="error" eventType="tier0_service_group_failover"] Context report: {"entity_id":"0000####-0000-####-0000-####0000####", "failover_reason":"Peer Node f0a8####-8639-####-9b3c-####e568#### Unreachable"...}

  •  NSX 1 - [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-agent" s2comp="nsx-monitoring" entId="0000####-0000-####-0000-####0000####" tid="1" level="WARNING" eventState="On" eventFeatureName="high_availability" eventSev="warning" eventType="tier0_service_group_reduced_redundancy"]

Environment

VMware NSX

Cause

This is a known issue during an upgrade when there is a change in the state of the T0-AA stateful configuration, causing it to become T0-AA stateless.

Resolution

Please follow the below workaround to resolve the issue:

1. Disconnect the Tier 1 (T1) gateways from the Tier 0 (T0) gateway.

2. Update the T0 gateway via API by modifying the stateful_services field in the payload as follows:

3. Connect the T1 gateways back to the T0 gateway.

4. Check on the Edge node if the interface-groups HA state becomes up, if the T1 AA state becomes up, and verify if traffic resumes.

5. If needed, restart the local controller on the Edge via nsxcli using the command: restart service local-controller.