Alarm for Registration/un-registration of third-party IO filter storage providers fails

Products

• VMware SDDC Manager
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0
• VMware vSphere ESXi

Issue/Introduction

• vCenter reports the warning "Registration/un-registration of third-party IO filter storage provider fails on a host.".
• IO Filter entries are not shown within vCenter Storage Providers.
• Issue could occur after moving hosts between Datacenters without maintenance mode.
• In the ESXi/ESX /var/run/log/iofiltervpd.log file, following entries will be seen:

yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: IOFilterVPControlHandleMessage:239:received command: addconf /usr/lib/vmware/vmiof/disk/vmwarevmcrypt-config.xml VMW_vmwarevmcrypt_1.0.0
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:247:/usr/lib/vmware/vmiof/disk/vmwarevmcrypt-config.xml: successfully loaded filter configuration.
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:248:FilterName: vmwarevmcrypt
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:249:FilterClass: encryption
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:250:FilterVersion: 1.0.0
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:251:FilterType: disk
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: LoadFilterCatalogs:191:Added 1 catalog(s) for filter vmwarevmcrypt
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: LoadConfiguration:158:Added configuration file: /usr/lib/vmware/vmiof/disk/vmwarevmcrypt-config.xml successfully.
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: AddConfiguration:258:Number of filters loaded: 1
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: IOFilterVPControlHandleMessage:239:received command: addconf /usr/lib/vmware/vmiof/disk/spm-config.xml VMW_spm_1.0.0
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:247:/usr/lib/vmware/vmiof/disk/spm-config.xml: successfully loaded filter configuration.
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:248:FilterName: spm
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:249:FilterClass: datastoreIoControl
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:250:FilterVersion: 1.0.230
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: ReadConfigFromFile:251:FilterType: disk
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: LoadFilterCatalogs:191:Added 8 catalog(s) for filter spm
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: LoadConfiguration:158:Added configuration file: /usr/lib/vmware/vmiof/disk/spm-config.xml successfully.
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: AddConfiguration:258:Number of filters loaded: 2
yyyy-mm-ddThh:mm:ssZ iofiltervpd[67088]: run:159:SSL Connection error 30 : SSL_ERROR_SSL error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

• VMware ESXi 7.x
• VMware ESXi 8.x
• VMware ESX 9.x

Cause

The IO Filter on the ESXi/ESX host is not correctly registered with the ESXi/ESX host certificate.

Resolution

Refresh the IO Filter Registration on the ESXi/ESX Host:

 

1. Before refreshing the IO filter registration, verify the Host Certificate and Certificate chain is OK on the ESXi/ESX. You can verify the certificate from vSphere client. Refer Manage Host Certificates by Using the VMware Host Client

2. SSH to the ESXi/ESX host as the root user.

3. Current cert files are located at the following locations:

/etc/vmware/ssl/rui.crt

/etc/vmware/ssl/castore.pem

Note: The following steps can be performed without placing the ESXi/ESX host into Maintenance Mode (MM). However, as a best practice and for added safety, it is recommended to carry out these steps while the ESXi host is in Maintenance Mode.

4. Take a backup of the current ESXi/ESX certificate file: cp /etc/vmware/ssl/castore.pem /etc/vmware/ssl/castore.pem.backup

5. Depending on the VMware ESXi host version, Manually refresh the certificates in the iofiltervp by running the following command:

a. VMware ESXi hosts 7.x/8.x

  /usr/lib/vmware/iofilter/bin/iofvp-ctrl-app -r

b. VMware ESX host 9.x

  /etc/init.d/hostd restart && /etc/init.d/vpxa restart

6. "Reset to Green" or acknowledge the alarm in vCenter after performing the above steps.