CPU on vCenter Server spikes and web client is not responding

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:
When using vCenter Server 6.0, you experience one or more of these symptoms:

• Cannot login or slow access to the vCenter Server using vSphere Web Client
• There is an alarm triggered with this description:

vsphere-client status changed from green to gray

 

You will also observed large number(more than a thousand) of "Waiting threads" in vsphere_client_virgo.log

 

Example:

 

Tomcat Request-Processing Threads Count: 10
Blocked threads: 0
Waiting threads: 14080

• You may also have problems accessing virtual machines via vCenter Server using PowerCLI.

 

 

Environment

VMware vCenter Server 6.0.x
VMware vCenter Server Appliance 6.0.x

Cause

CEIP VLSI clients are not properly closed.

Resolution

This is a known issue affecting the VMware vCenter Server 6.0.

 

This issue is resolved in VMware vCenter Server 6.0 Update 3, available at
For more information, see the

To workaround this issue remove customer experience improvement plugin on vCenter Server.

 

 

vCenter Server on Windows

1. Start a command line console.

 

2. Stop the vSphere Web Client service by running command:

 

net stop vspherewebclientsvc /y
 

The /y parameter stops the vmware-perfcharts service which depend on vspherewebclientsvc.

 

3. Remove the contents of the vSphere Web Client work directory by running command:

 

rmdir "C:\Program Files\VMware\vCenter Server\WebClient\server\work" /s /q

 

4. Remove the contents of the pickup directory by running command:

 

del "C:\Program Files\VMware\vCenter Server\virgo\server\pickup\*" /q

 

5. Back up the following files that are located in C:\Program Files\VMware\vCenter Server\WebClient\plugin-packages\vsphere-client\plugins\:

 

telemetry-service-6.0.0.jar

telemetry-ui-war-6.0.0.war

phonehome-collector-ui-war-6.0.0.war

cis-data-service-cmc-6.0.0.jar

 

6. Remove the following files that are located in C:\Program Files\VMware\vCenter Server\WebClient\plugin-packages\vsphere-client\plugins\:

 

telemetry-service-6.0.0.jar

telemetry-ui-war-6.0.0.war

phonehome-collector-ui-war-6.0.0.war

cis-data-service-cmc-6.0.0.jar

 

7. Start the following vCenter services by running command:

 

net start vspherewebclientsvc

net start vmware-perfcharts

 

vCenter Server Appliance

 

To work around the issue open an SSH session with the vCenter Server Appliance. See Enable or Disable SSH and Bash Shell Access in the vSphere Documentation Center.

Note: The Customer Experience Improvement Program Plugin is removed from running through these steps!

 

1. Stop the vSphere Web Client service by running command:

 

service vsphere-client stop

 

2. Remove the contents of the vSphere Web Client work directory by running command:

 

rm -rf /usr/lib/vmware-vsphere-client/server/work/*

 

3. Remove the contents of the pickup directory by running command:

 

rm /usr/lib/vmware-virgo/server/pickup/*

 

4. Back up the following files that are located in /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/:

 

telemetry-service-6.0.0.jar

telemetry-ui-war-6.0.0.war

phonehome-collector-ui-war-6.0.0.war

cis-data-service-cmc-6.0.0.jar

 

5. Remove the following files that are located in /usr/lib/vmware-vsphere-client/plugin-packages/vsphere-client/plugins/:

 

telemetry-service-6.0.0.jar

telemetry-ui-war-6.0.0.war

phonehome-collector-ui-war-6.0.0.war

cis-data-service-cmc-6.0.0.jar

 

6. Start the following vCenter service by running:

 

service vsphere-client start