Error Code: 500016 when trying to view group members or checking the DFW rule status on the Global Manager (Error: I/O error)
search cancel

Error Code: 500016 when trying to view group members or checking the DFW rule status on the Global Manager (Error: I/O error)

book

Article ID: 313769

calendar_today

Updated On:

Products

VMware NSX VMware vDefend Firewall

Issue/Introduction

  • NSX Federated environment.

  • Recently replaced certificates or expired certificates on Local Manager and/or Global Manager.

  • An error stating  General Error  or  Error: unable to onboard remote host due to incorrect thumbprint. (Error code: 500340) may be present on one ore more Local Managers on the Location Manager page of the Global Manager UI.

  • When trying to view members on the Global Manager you may see similar errors in the NSX UI like:

Error: I/O error on GET request for "https://<nsx-mgr-ip>/policy/api/v1/global-infra/domains/default/groups/ipset-########-####-####-####-########/members/ip-addresses": UUID; nested exception is javax.net.ssl.SSLHandshakeException: UUID (Error code: 500016)

    • Note: The error above is not exclusively seen for IP Addresses, the same error can be seen for Virtual Machines, NSX Segments, etc.

  • When trying to check the status for DFW rules in the Global NSX UI, you may get this Unknown status for the problematic LM:  

  • An I/O error on GET request may be present when attempting to check rule statistics:

 

  • When selecting 'Generate BGP Summary,' an error message similar to the following may be observed:

    Error: I/O error on GET request for "https://####-########.#######.com/policy/api/v1/global-infra/tier-0s/NA-##-##-###-###-Exit/locale-services/NA-S2-####-########.#######.com/bgp/neighbors/status": #######################################################; nested exception is javax.net.ssl.SSLHandshakeException: ####################################################### (Error code: 500016)

Environment

VMware NSX 4.x
VMware NSX-T Data Center
VMware NSX Federation

Cause

After certificate replacement on NSX Local Manager nodes, the thumbprint for site locations may not update automatically on the Global Manager, causing TLS handshake failures between the Global Manager and any Local Managers with outdated certificate thumbprint data.

Resolution

Verify the thumbprint is correct for both Local Manager sites.

  • This can be checked on the Global Manager, System Location Manager, under the site location Actions > Edit Settings and Check Version Compatibility. If it fails, the thumbprint will need to be updated.

To update the Local Manager sites' API thumbprint information:

  1. Log into the VIP assigned local manager on to each local manager site as the admin user and run the command 'get certificate api thumbprint' .

  2. Copy the thumbprint and paste it in the the Thumbprint text box under the site location Actions > Edit Settings.


  3. Enter the Password.

  4. Click on "Check Version Compatibility" again and make sure it succeeds.
    • It will display Version information and a green checkmark if it is successful.
    • A message stating that the thumbprint does not match or that the credentials were incorrect/locked will be displayed if not.

  5. Save the updated settings.

Additional Information

Powered by Wolken Software