CPU on vCenter Server spikes to almost 100% and web client crashes

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

Web client is too slow and provides this error:

 

Error processing a query spec java.lang.RuntimeException: java.lang.OutOfMemoryError: PermGen space

• On restarting the Web Client Service web client works for 5 minutes
• CPU usage on the vCenter Server Spikes almost to 100% and web client crashes
• Removing all the plug-ins or without any plugins registered, web client works fine

vsphereclient virgo logs:

[2016-06-02T12:36:53.858-04:00] [INFO ] or[StandardEngine[Catalina]] com.vmware.vise.security.SessionListener sessionDestroyed invoked for sessionId 100003

[2016-06-02T12:37:08.962-04:00] [INFO ] or[StandardEngine[Catalina]] ######## 100003 200001 com.vmware.vise.security.DefaultAuthenticationProvider

Beginning logout for sessionId 100003, clientId 200001 [2016-06-02T12:38:32.407-04:00] [ERROR] or[StandardEngine[Catalina]] ######## 100003 200001 com.vmware.vise.util.concurrent.WorkerThread

ContainerBackgroundProcessor[StandardEngine[Catalina]] terminated with exception: java.lang.OutOfMemoryError: PermGen space

Environment

VMware vCenter Server 6.0.x

Resolution

Perform the steps below:

1. Increase the memory of vCenter Server virtual machine (If it is already high, skip to step 2).
2. Increase the heap size of the web client running the command and monitor if it helps. If increasing heap size move to step 3.
    
   • On vCenter Server Appliance, run:
     cloudvm-ram-size -C XXX vsphere-client (where XXX is the desired heap size in MB).
      
   • In Windows, locate the file C:\Program Files\VMware\vCenter Server\visl-integration\usr\sbin)
     \cloudvm-ram-size.bat and run:
     
     cloudvm-ram-size.bat -C XXX vspherewebclientsvc (where XXX is the desired heap size in MB).
      
   • After changing the maximum heap size, you’ll need to restart the vSphere Web Client service for the change to take effect.
      
3. Increase the PermGen space if more plugins are registered to vCenter.
   
   C:\ProgramData\VMware\vCenterServer\cfg and open the file service-layout.mfx in notepad and change the MaxPermMB.

In vCenter server appliance:

Navigate to  /etc/vmware and open the file service-layout.mfx.

Notes:

• Changing the PermGen space without a valid reason and the error logging terminated with exception: java.lang.OutOfMemoryError: PermGen space in virgo logs is not recommended
• When the number of Plugins register to vCenter are more, then the PermGen space can be increased if there are performance issues seen on the web-client .