Storage providers show as offline on vCenter server | sps.log file generates excessive entries

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Storage providers show as offline in VCSA > Configure > Storage Providers

• sps.log file is generating excessive amount of messages and below log entries to be seen:

value = No VASA Provider for schema namespace (vmwarevmcrypt) found.

Exception occurred while checking if host is inSync.

Exception occurred while syncing host.

Environment

VMware vCenter Server 7.0.x

Cause

Storage providers are in offline state, which causes failed scans and results in a bloated sps.log file.

Resolution

This process will reset the VASA providers:

Note: Please take an offline snapshot and/or backup of all nodes in the SSO domain. Do not skip this step.

1. Stop the Storage Provider Service: service-control --stop vmware-sps

2. Delete the SMS certificate from VECS: /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store sms --alias sms_self_signed

3. Start the SPS service: service-control --start vmware-sps

   This will have the effect of SMS self-signed certificate re-generated with sha256withRSAEncryption signing algorithm and this certificate will now be FIPS compliant. Regeneration of the cert will also cause all the storage providers to go offline as it's a new cert and no longer trusted; meaning step 5 will unregister them and clear them out.

4. After step 3 above, Wait for some time for SPS service to get into initialized state and health status is GREEN. Use the wget command above to check. Initialization can take a long time.

5. Once after storage providers list is updated and providers are listed (there will be providers listed with offline/disconnected state), run the python script "unreg_vasa.py" and capture the output (needed only in case if the steps doesn't resolve the issue),

      python unreg_vasa.py -s <VC_IP_ADDRESS>

      NOTE: <VC_IP_ADDRESS> above need to be replaced with actual values.

      NOTE: The script waits 5 seconds between unregistering each VP. Please wait for it to finish

6. Restart vmware-sps service: vmon-cli -r sps

      This will have the effect of all the IOFilter VPs that was unregistered in step 5 to get registered again.