Edge showing MPA Connectivity Down after storage/vSAN issues

Products

• VMware NSX

Issue/Introduction

• If any storage related issues happen, and Edge status on NSX shows as "MPA Connectivity Down", this indicates that the connectivity from Edge to Manager is not happening on port 1234. The RPC (Remote Procedure Call) Channel that the NSX manager uses to communicate with Edge on port 1234 that is the MP (Management Plane) Channel on port 1234 is not working:

         

• Check the connectivity from NSX Edge to NSX Manager:
  • From NSX Edge run nc -zv <NSX-Manager-IP> 1234 
  • Repeat this towards all the managers
    • Connectivity is successful:
    • 
• Then go to NSX Edge CLI (admin) and run get managers
  • Managers are not listed even though the connectivity on Port 1234 works
    • 

From the Edge logs we can see: 

/var/log/syslog:

[WARNING] No aph found in appliance-info.xml

Also one more error may be seen:

[WARNING] Could not read a valid uuid from /etc/vmware/nsx/host-cfg.xml

Environment

VMware NSX

Cause

Few times storage/datastore issues may cause files to be corrupt on Edge transport node. If the issue continues even after fixing the file system (using this KB), the file /etc/vmware/nsx/appliance-info.xml is still empty for some Edges and also for some nodes this file /etc/vmware/nsx/host-cfg.xml may have missing its own UUID.

Resolution

Workaround:

1. When checked the file at location: cat /etc/vmware/nsx/appliance-info.xml, it was empty but on a good Edge it has the appliance proxies info (with all the 3 manager details in it), meaning all the manager IPs with port 1234 and their cert details are present in that file. Copy the details from good Edge appliance-info.xml to the bad Edge which does not have it.

2. Restart nsx-proxy service using: /etc/init.d/nsx-proxy restart to take the changes into effect.

3. Once this is completed, status of the Edge on NSX manager shows as 'Success', and also get managers on admin CLI shows all managers and status as 'Connected'.

4. If in case the above command's output shows all managers but is in state Standby, please check the logs (/var/log/syslog) and we can see an error in hosts-cfg.xml that Edge's UUID is missing

Copy the UUID from NSX manager UI and place it in the file hosts-cfg.xml and restart nsx-proxy should fix the MPA connectivity issue.