NSX-T Federation Deployment: Connection from Local Manager to the Global Manager is broken in the NSX Federation after replacing the Local Managers mp-cluster certificate

Products

• VMware NSX

Issue/Introduction

• On the NSX-T Global Manager GUI, when the user navigates to System > System Overview

The user sees that the "Appliances", "Latest Backup", "Automatic Backups" and "Remote Tunnel Endpoint Status" status on one/all Local Managers are "Not available"

• When the user tries to re-register the site, the user observers following error on GUI:

Error: Communication error occurred with reason 500 Internal Server Error: "{"error_code": 37000, "error_message": "Site manager error. [SM180] Unable to retrieve certificates from trust manager.", "module_name": "node-services"}" (Error code: 530039)

• You may encounter log entries such as the below on the Global Manager:

/var/log/gmanager/gmanager.log

2022-xx-xxTxx:xx:xx.xxxZ nsx-t-mgr NSX 4938 POLICY [nsx@6876 comp="global-manager" level="WARNING" reqId="xxxxxx-xxxx-xxxx-xxxxx-xxxxxxx" subcomp="global-manager" username="admin"] Failed to on-board with exception 500 Internal Server Error: "{"error_code": 37000, "error_message": "Site manager error. [SM180] Unable to retrieve certificates from trust manager.", "module_name": "node-services"}"

• You may encounter log entries such as the below on the Global and Local Manager:

/var/log/proton/nsxapi.log
2022-xx-xxTxx:xx:xx.xxxZ  INFO http-nio-127.0.0.1-7440-exec-19 NsxBaseRestController 10551 SYSTEM [nsx@6876 comp="global-manager" level="INFO" subcomp="manager"] Error in API /nsxapi/api/v1/trust-management/site-certificates caused by exception com.vmware.nsx.management.common.exceptions.ObjectNotFoundException:  {"moduleName":"common-services","errorCode":600,"errorMessage":"The requested object : Certificate/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxx could not be found. Object identifiers are case sensitive."}
 

• Cosmetic issue with no data plane impact

Environment

VMware NSX-T Data Center 3.x
VMware NSX-T Data Center
VMware NSX-T

Cause

In the NSX-T Federation, The LM to GM connectivity broke after replacing the certificate on LM an VIP

The issue is triggered when the certificate referenced by a site-certificate can't be found.

Resolution

Upgrade to NSX-T version 4.1.0 or later

Additional Information

Certificates for NSX Federation