vmware-perfcharts service failed to start after upgrade vCenter Server
Article ID: 326213
Updated On:
Products
VMware vCenter Server
Issue/Introduction
Symptoms:
- You upgraded the vCenter Server to 7.0 U2d.
- The upgrade process was finished but vmon failed to start the vmware-perfcharts service with the messages in /var/log/vmware/applmgmt/PatchRunner.log:
Performing start operation on profile: ALL...
stderr=Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start perfcharts services. Error: Operation timed out
stderr=Service-control failed. Error: Failed to start services in profile ALL. RC=1, stderr=Failed to start perfcharts services. Error: Operation timed out
- In the /var/log/vmware/perfcharts/vmware-perfcharts-runtime.log.stderr it may contain below messages:
23-Sep-2021 15:16:18.401 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server} Setting property 'port' to '${base.shutdown.port}' did not find a matching property.
23-Sep-2021 15:16:19.092 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'port' to '${bio.http.port}' did not find a matching property.
....
23-Sep-2021 15:16:38.541 SEVERE [main] org.apache.catalina.core.StandardService.startInternal Failed to start connector [Connector[HTTP/1.1-auto-1]]
org.apache.catalina.LifecycleException: The connector cannot start since the specified port value of [-1] is invalid
23-Sep-2021 15:16:19.092 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'port' to '${bio.http.port}' did not find a matching property.
....
23-Sep-2021 15:16:38.541 SEVERE [main] org.apache.catalina.core.StandardService.startInternal Failed to start connector [Connector[HTTP/1.1-auto-1]]
org.apache.catalina.LifecycleException: The connector cannot start since the specified port value of [-1] is invalid
Environment
VMware vCenter Server 7.0.x
Cause
The tomcat could not read the two system properties base.shutdown.port=-1 and bio.http.port=13080 which defined in /usr/lib/vmware-perfcharts/tc-instance/conf/catalina.properties. The file may be corrupted or incorrect permission.
Resolution
To resolve the issue, the first step is to check if the catalina.properties file is corrupted or not.
1. Open SSH to VCSA, and go to the /usr/lib/vmware-perfcharts/tc-instance/conf/
2. Open the file catalina.properties to verify if its content is normal, specially the last two settings existing:
base.shutdown.port=-1
bio.http.port=13080
3. If the two settings are not here, manually add them to the end of the file and save it.
4. If the file is empty or corrupted, manually copy the same file from a good vCenter server to replace it.
If the content of the file is correct, secondly check its permission if correct:
1. Under /usr/lib/vmware-perfcharts/tc-instance/conf/ input the command: ls -l
2. All files in this path should have the same permission liking this:
root@vc1 [ /usr/lib/vmware-perfcharts/tc-instance/conf ]# ls -l
total 224
-rwx------ 1 perfcharts cis 12874 Sep 27 14:04 catalina.policy
-rwx------ 1 perfcharts cis 8531 Sep 27 15:44 catalina.properties
-rwx------ 1 perfcharts cis 123 Sep 27 14:04 context.xml
-rwx------ 1 perfcharts cis 1149 Sep 27 14:04 jaspic-providers.xml
-rwx------ 1 perfcharts cis 2313 Sep 27 14:04 jaspic-providers.xsd
-rwx------ 1 perfcharts cis 4069 Sep 27 14:04 logging.properties
-rwx------ 1 perfcharts cis 2776 Sep 28 01:30 server.xml
-rwx------ 1 perfcharts cis 2164 Sep 27 14:04 tomcat-users.xml
-rwx------ 1 perfcharts cis 2558 Sep 27 14:04 tomcat-users.xsd
-rwx------ 1 perfcharts cis 171510 Sep 27 14:04 web.xml
3. In some cases the permission of catalina.properties is incorrect liking this:
-rwx------ 1 root root 8531 Sep 27 15:44 catalina.properties
4. Change the permission of this file:
# chown perfcharts catalina.properties
# chgrp cis catalina.properties
After confirm above items, start the vmware-perfcharts service again:
# service-control --start vmware-perfcharts
1. Open SSH to VCSA, and go to the /usr/lib/vmware-perfcharts/tc-instance/conf/
2. Open the file catalina.properties to verify if its content is normal, specially the last two settings existing:
base.shutdown.port=-1
bio.http.port=13080
3. If the two settings are not here, manually add them to the end of the file and save it.
4. If the file is empty or corrupted, manually copy the same file from a good vCenter server to replace it.
If the content of the file is correct, secondly check its permission if correct:
1. Under /usr/lib/vmware-perfcharts/tc-instance/conf/ input the command: ls -l
2. All files in this path should have the same permission liking this:
root@vc1 [ /usr/lib/vmware-perfcharts/tc-instance/conf ]# ls -l
total 224
-rwx------ 1 perfcharts cis 12874 Sep 27 14:04 catalina.policy
-rwx------ 1 perfcharts cis 8531 Sep 27 15:44 catalina.properties
-rwx------ 1 perfcharts cis 123 Sep 27 14:04 context.xml
-rwx------ 1 perfcharts cis 1149 Sep 27 14:04 jaspic-providers.xml
-rwx------ 1 perfcharts cis 2313 Sep 27 14:04 jaspic-providers.xsd
-rwx------ 1 perfcharts cis 4069 Sep 27 14:04 logging.properties
-rwx------ 1 perfcharts cis 2776 Sep 28 01:30 server.xml
-rwx------ 1 perfcharts cis 2164 Sep 27 14:04 tomcat-users.xml
-rwx------ 1 perfcharts cis 2558 Sep 27 14:04 tomcat-users.xsd
-rwx------ 1 perfcharts cis 171510 Sep 27 14:04 web.xml
3. In some cases the permission of catalina.properties is incorrect liking this:
-rwx------ 1 root root 8531 Sep 27 15:44 catalina.properties
4. Change the permission of this file:
# chown perfcharts catalina.properties
# chgrp cis catalina.properties
After confirm above items, start the vmware-perfcharts service again:
# service-control --start vmware-perfcharts
Feedback
Yes
No
Powered by
