VMware vSphere Profile-Driven Storage Service Health Alarm on the vCenter server

Products

• VMware vSphere ESXi

Issue/Introduction

• "VMware vSphere Profile-Driven Storage Service health alarm" is triggered against the vCenter Server.
  
  
• In the vCenter Server Appliance Management Interface (VAMI), the Storage Profile Service (SPS) status may show "Healthy with warnings."
  
  
• When checking the health status via the command line on the vCenter Server Appliance (VCSA), the following command:
  
  curl http://localhost:22000/sms/HealthStatus, below is the output:

<? xml version="1.0" encoding="UTF-8" standalone="yes"?><healthStatus schemaVersion="1.0" xmlns="http://www.vmware.com/cis/cm/common/jaxb/healthstatus"><status>YELLOW</status><message messageKey="com. vmware.vim. sms. ServiceFailure" defaultMessage="Problem encountered while communicating to {0]. Please check the service accordingly."><param xsi:type="xs: string" xmlns :xs="http: //www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">VMware Certificate Service</param></message><</healthStatus>

• Restarting only the vmware-sps service shows the health status to be GREEN for a while, but the alarm reappears.
  
  
• SMS certificates are valid.
  
  
• In the vCenterserver /var/log/vmware/vmware-sps/sps.log shows the below :

<timestamp> Wa(03) host-2484 <sps> Service api-health command's stderr: Exception while retrieving health xml from url http://localhost:22000/sms/HealthStatus. Exception: <urlopen error [Errno 111] Connection refused>
<timestamp> Wa(03) host-2484 <sps> Service api-health command's stderr: Exception while retrieving health xml from url http://localhost:22000/sms/HealthStatus. Exception: <urlopen error [Errno 111] Connection refused>

Environment

VMware vCenter Server 8.0.x

Cause

This issue occurs due to a transient communication failure between the Storage Profile Service and the internal VMware Certificate Service. 
This prevents the SPS from validating the health of the storage monitoring components even if the underlying certificates are not expired.

Resolution

To resolve this issue, restart the Storage Profile Service and the Certificate manager service to re-establish the communication session.

1. Log in to the vCenter Server Appliance via SSH using root credentials.
   
   
2. Restart the vmware-sps and the vmcad service by running: 
   
   service-control --restart vmware-sps
service-control --restart vmcad
   
   
3. After the service has successfully restarted, wait 2–3 minutes and verify the health status again: curl http://localhost:22000/sms/HealthStatus (The output should now be GREEN)
   
   
4. In the vSphere Client, locate the alarm on the vCenter Server object and select Reset to Green.

Additional Information

In case the issue still persists, raise a ticket with the Broadcom Support team.