HTTP Error 503: Service not available while accessing Product UI in vRealize Operations Manager

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Accessing Product UI in vRealize Operations Manager fails with the error:
  
  vRealize Operations Manager is not available. You will be redirected when the service is available.
Log into the Admin UI to configure, check, or repair the cluster.

• You can access Product UI from another node in the cluster.
• In the /usr/lib/vmware-vcops/tomcat-web-app/logs/catalina.yy-mm-dd.log file, you see entries similar to:
  
  org.apache.coyote.AbstractProtocol init
SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-bio-8443"]
java.io.IOException: Keystore was tampered with, or password was incorrect
  
  

   

   

Cause

This issue occurs when:

• The directory tomcat-web-app is missing.
• The symbolic link for vcops-web-ent file is not pointing to /usr/lib/vmware-vcops/tomcat-web-app/webapps/ui.
• The keystore and truststore are tampered with or there is a mismatch in catalina.properties with configured values in storePass.properties.

Resolution

To resolve this issue, replace the tomcat-web-app directory.

Note:

1. VMware recommends to take a snapshot of all vRealize Operations nodes in the cluster before performing these steps
2. If the steps are not performed correctly, you may encounter an unsupported configuration.

 

To replace the tomcat-web-app directory:

1. Create a copy of a tomcat-web-app directory from any of the working Master or Data nodes:
   
   For example:
   
   cd /usr/lib/vmware-vcops
tar czpf tcwebapp.tgz tomcat-web-app
   
   Note: The p flag preserves file permissions.
   
   
2. Copy the zipped file to the node having the issue (Master or Data) and run these commands:
   
   Note: Assuming tcwebapp.tgz is copied to /tmp

cd /usr/lib/vmware-vcops
rm -rf tomcat-web-app
tar xzpf /tmp/tcwebapp.tgz
rm -rf tomcat-web-app/work/*
rm -rf tomcat-web-app/temp/*
   
   
3. Modify the /usr/lib/vmware-vcops/tomcat-web-app/conf/catalina.properties
   If the login page appears then you can access the Product UI.

Additional Information