MANAGER and HTTPS services are down causing the management cluster to be in degraded state
search cancel

MANAGER and HTTPS services are down causing the management cluster to be in degraded state

book

Article ID: 322438

calendar_today

Updated On:

Products

VMware NSX Networking

Issue/Introduction

Symptoms:
  • You are using NSX 4.1 and the NSX-UI is inaccessible.
  • In the nsxcli cli as the admin user, when running the command get cluster status, we see the HTTPS and MANAGER service are DOWN:
Group Type: MANAGER
...
    2c231d42-3fd9-8978-2648-88a7787ef6d0       nsx-mngr-01.corp.local                     192.168.120.1    DOWN
...
Group Type: HTTPS
...
    2c231d42-3fd9-8978-2648-88a7787ef6d0       nsx-mngr-01.corp.local                     192.168.120.1    DOWN
  • vMotion is blocked as the nsxa agent is down on the transport node(s).
  • In /var/log/proton/nsxapi.log in the NSX manager logs you see the following NullPointerException:
Caused by: java.lang.NullPointerException
    at com.vmware.nsx.management.security.firewall.service.FirewallPropertyCache._refresh(FirewallPropertyCache.java:55) ~[?:?]
    at com.vmware.nsx.management.security.firewall.service.FirewallPropertyCache.init(FirewallPropertyCache.java:44) ~[?:?]
  • Running the following GET API call from root of an SSH session to the NSX manager you will see an entry as follows, with the key equal to operation:
root@manager1:# curl -k -u 'admin:<password>' -H 'X-NSX-Username:admin' -H 'Content-Type:application/json' -X GET http://127.0.0.1:7440/nsxapi/api/v1/firewall/troubleshoot/properties 
...
    "key" : "ipfix_update_dest_port",
    "value" : "success#2023-05-21T01:00:58.726"
  }, {
    "key" : "operation", 
    "value" : "ipfix_update_dest_port"
  }, {
    "key" : "popularityindextimeunitms",
    "value" : "3600000"
...
NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.


Environment

VMware NSX 4.0.0.1

Cause

This issue is caused when the following internal POST API call is run:
root@manager1:# curl -k -u 'admin:<password>' -H 'X-NSX-Username:admin' -H 'Content-Type:application/json' -X POST http://127.0.0.1:7440/nsxapi/api/v1/firewall/troubleshoot/properties
This API call is for troubleshooting purposes only and should only be run under VMware GSS guidance.

Resolution

This is a known issue impacting VMware NSX.

Workaround:
To work around this issue, contact VMware Support and note this Article ID (92994) in the problem description. For more information, see How to Submit a Support Request.