Certain IOFIlter Providers are showing as offline

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

To get the IOfilter providers online.

 

Symptoms:

• Certain IOFIlter Providers are showing as offline
• In the vCenter log: /var/log/vmware/vmware-sps/sps.log you will see entries similar to the below:

• [YYYY-MM-DDTHH:MM:SS]  [pool-10-thread-5] ERROR opId=sps-Main-101761-246 com.vmware.vim.sms.provider.vasa.alarm.AlarmDispatcher -
  [YYYY-MM-DDTHH:MM:SS] [pool-10-thread-1] ERROR opId=sps-Main-101761-246 com.vmware.vim.sms.provider.vasa. alarm. AlarmDispatcher - Error: org.apache.axis2.AxisFault: self signed certificate occured as provider: https://<ESXi-A FQDN>:9080/version.xml:9080/version.xml is
  offline
  [YYYY-MM-DDTHH:MM:SS]  [pool-10-thread-4] ERROR opId=sps-Main-101761-246 com.vmware.vim.sms.provider.vasa.alarm.AlarmDispatcher - Error: org.apache.axis2.AxisFault: self signed certificate occured as provider: https://<ESXi-C FQDN>:9080/version.xml:9080/version.xml is
  offline

  Error: org.apache.axis2.AxisFault: self signed certificate occured as provider: https://<ESXi-A FQDN>:9080/version.xml is offline

• The sps service is showing as green when running the below command:

  # wget localhost:22000/sms/HealthStatus ; cat HealthStatus ; rm HealthStatus

• Running the command netstat -pan | grep sps does not show any entries in CLOSE_WAIT for port 9080.

 

Note: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware vCenter Server 8.x

VMware vSphere ESXi 8.x

VMware vCenter Server 7.x

VMware vSphere ESXi 7.x

VMware vCenter Server 6.7

VMware vSphere ESXi 6.7

Resolution

NOTE: Before taking the steps below, please ensure that a fresh snapshot or backup of the vCenter Server Appliance (VCSA) was created. In case the VCSA is in an Enhanced Linked Mode replication with other vCenter Servers, please be aware that offline snapshots of all ELM replication partners need to be taken, and need to be restored in case of a rollback.

Also please ensure that the IOFilter provider URLs can be reached from the vCenter, by running wget against them. Once this was verified, please apply one of the options below:

 

Solution 1 :

1. Log in to the following link:

   https://<vCenter-IP>/sms/mob

2. Click on QueryStorageManager link
3. Click on Invoke method, then on the "storageManager" link in the Method Invocation Result
4. Click on the QueryProvider link and Invoke method
5. Click through the vasaProvider-XXX links in the Method Invocation Results one by one
6. For each vasaProvider selected click on QueryProviderInfo link and Invoke Method
7. Cross reference the result for "uid" field and note it down for each "url" entry that matches from the url's mentioned in the vCenter Storage Provider pane as showing offline. For example:

   (sms.provider.VasaProviderInfo) { dynamicType = null, dynamicProperty = null, uid = b9ac8d33-fb57-4c98-8c42-be36a633f1ce

8. Go Back to the QueryStorageManager link (see step 1 and step 2)
9. Click on UnregisterProvider_Task link
10. Enter one of the the provider uid's found in step 6 and click on the invoke method.
11. Repeat steps 7 to 10 for each of the IOFilter providers shown as offline
12. Restart the SPS service with these commands to reregister the IOFilter providers:

    # service-control --stop vmware-sps
    # service-control --start vmware-sps

    
    
    

 

Solution 2 :

1. Open PowerCLI
2. Connect PowerCLI to the vCenter Server:

   # Connect-VIServer --server <vCenter-FQDN>

3. Run the following command to remove offline IOfilters:

   # Get-VasaProvider | Where-Object {$_.Namespace -eq 'IOFILTERS' -and $_.Status -like 'offline'} | Remove-VasaProvider

4. In vSphere Client, go to vCenter > Configure > Storage Providers.
5. Click on “Synchronize Storage Providers”.