VM Edit settings in vSphere Client unable to load due to high thread consumption on vmware-sps.

Products

• VMware vCenter Server

Issue/Introduction

The vCenter Server Appliance (VCSA) experiences significant performance degradation, When trying to edit any VM the page for edit settings goes blank or takes time to load.

1. The issue is temporarily resolved by restarting the vmware-vpxd and vmware-sps services.
2.  Log Snippet 

   • /var/log/vmware/vmware-sps/sps.log
     
     YYYY-MM-DDTHH:MM:SS.593Z [pool-27-thread-5] ERROR opId=sps-Main-655405-438 com.vmware.vim.sms.provider.vasa.alarm.AlarmDispatcher - Error: org.apache.axis2.AxisFault: self-signed certificate occured as provider: https://<ESXi-Hostname>:9080/version.xml is offline
     YYYY-MM-DDTHH:MM:SS.593Z[pool-27-thread-2] ERROR opId=sps-Main-655405-438 com.vmware.vim.sms.provider.vasa.alarm.AlarmDispatcher - Error: org.apache.axis2.AxisFault: self-signed certificate occured as provider: https://<ESXi-Hostname>:9080/version.xml is offline
     

3. vimtop command shows high memory consumption for these services the vsphere-ui, vpxd, and sps (Storage Profile Service) services are using high threads. 

Environment

vCenter  Server Appliance

Cause

This occurs because Storage I/O Filter (iofilter) providers for decommissioned or disconnected hosts remain registered in the vCenter database. The sps service continuously attempts to poll these offline providers. This constant polling leads to:

1. Thread Exhaustion: The service consumes all available threads trying to reach unreachable endpoints.
2. Memory Leak: The request queue grows indefinitely, causing the JVM heap to reach its limit.

Resolution

Step 1: Identify Offline I/O Filter Providers

1. Log in to the vSphere Client.
2. Navigate to the vCenter Server object in the inventory.
3. Select the Configure tab.
4. Under Storage, select Storage I/O Filters.
5. Identify any filters where the status is "Offline" or associated with hosts that have been decommissioned or permanently removed.

Step 2: Remove Stale I/O Filter Providers

• To fix offline ioFilter providers, please follow the instructions in Solution 1 or Solution 2; Certain iofilter providers are showing as offline

Important: Ensure you have a valid file-based backup of the vCenter Server before proceeding with manual provider removal or database modifications.

Step:3 Restart service

Once the stale providers are removed, the sps service must be restarted to clear the thread queue and re-register the storage I/O filter for existing ESXi hosts.

1. Access the vCenter Server via SSH or the Appliance Shell.
2. Restart the Storage Profile Service:
   • service-control --restart vmware-sps

Step 4: Verification

• Monitor the vCenter Server memory usage using top or vimtop within the VCSA shell.
• Confirm that the thread count for the sps process has stabilized.
• Validate that the Storage I/O Filters view no longer displays the decommissioned providers.

Additional Information

• Workaround for loadVasaProviders issue for IOFILTER VP's only
• Stopping, Starting or Restarting VMware vCenter Server Appliance services