NSX Manager UI is not accessible with "HTTP Status 404" error after reboot step from KB 87099
search cancel

NSX Manager UI is not accessible with "HTTP Status 404" error after reboot step from KB 87099

book

Article ID: 327348

calendar_today

Updated On:

Products

VMware NSX Networking

Issue/Introduction

The purpose of this article is to inform that the symptom can be seen where NSX manager UI is not accessible with reboot from log4j vulnerability KB  87099.
The issue here is not caused by log4j patch  and is instead due to corrupted ROOT.war file

Symptoms:

#With reboot step from KB https://ikb.vmware.com/s/article/87099, NSX manager UI not accessible. 

#SSH access to NSX manager works fine.

#NSX Manager UI is not accessible. "HTTP Status 404" error is seen 

#NSX plugin in vCenter shows "No NSX managers available. Verify current user has role assigned on NSX Manager
 

nsx-wrapper logs show below JM launching and connector failure : 

STATUS | wrapper | 2021/12/18 14:15:48 | Launching a JVM...

INFO | jvm 1 | 2021/12/18 14:15:48 | WrapperManager: Initializing...

 

Following error was seen on manager logs 

appliance_mgmt/appmgmt-wrapper.log:1072:INFO | jvm 1 | 2021/12/18 17:24:26 | 18-Dec-2021 17:24:26.283 SEVERE [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/usr/appmgmt-webserver/webapps/ROOT.war]

appliance_mgmt/catalina.2021-12-18.log:3485:18-Dec-2021 12:57:38.548 SEVERE [localhost-startStop-2] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/usr/appmgmt-webserver/webapps/ROOT.war]

appliance_mgmt/catalina.2021-12-18.log:3589:18-Dec-2021 14:15:52.003 SEVERE [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/usr/appmgmt-webserver/webapps/ROOT.war]

# As a part of log4j patching backup of the ROOT.war files is taken in /home/secureall/secureall/log4j-backup/

# verify the size of log4j-backup file with a working setup ROOT.war files 

# Also extracting the corrupted war files in backup folder will throw error. 

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment


Cause

Root cause for the NSX manager UI is not log4j Patch, instead corrupted ROOT.war file

INFO | jvm 1 | 2021/12/18 14:15:52 | 18-Dec-2021 14:15:52.023 SEVERE [WrapperStartStopAppMain] org.apache.catalina.core.StandardService.startInternal Failed to start connector [Connector[AJP/1.3-8001]]

INFO | jvm 1 | 2021/12/18 14:15:52 | Caused by: java.lang.IllegalArgumentException: The AJP Connector is configured with secretRequired="true" but the secret attribute is either null or "". This combination is not valid.

Resolution

Script in KB 87099 has been updated with fix to detect any ROOT.war file corruption


Workaround:
Restore NSX Manager from backup with uncorrupted ROOT.war files and then apply the latest script.

Additional Information

Impact/Risks:
NSX manager UI not accessible. NSX not available from VC plugin as well for any changes.