Error: "IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified" when registering or unregistering third-party IO filter storage providers on a host

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• ESXi summary page shows the following in the Issues and Alarms section:
  
  Registration/unregistration of third-party IO filter storage providers fails on a host
ESXi VASA client certificate provision has failed
  
  
  
  
• Running the command "journalctl -b" on vCenter Server will show:
  
  [User logged event: Registration/unregistration of third-party IO filter storage providers fails.]
[User logged event: Registration/unregistration of third-party IO filter storage providers fails on a host on several ESXi hosts.]
  
  
• Status of IOFilter for the respective hosts are shown as "Offline" under Storage Provider Tab (vCenter > Configure > Storage Providers)
  
  
  
• Iofiltervpd service logs on the ESXi repeatedly shows the following error message while verifying the certificate:|
  
  ESXi - /var/run/log/iofiltervpd.log
  YYYY-MM-DDTHH:MM:SSZ iofiltervpd[2098637]: IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified
YYYY-MM-DDTHH:MM:SSZ iofiltervpd[2098637]: IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified
YYYY-MM-DDTHH:MM:SSZ iofiltervpd[2098637]: IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified
YYYY-MM-DDTHH:MM:SSZ iofiltervpd[2098637]: IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified
YYYY-MM-DDTHH:MM:SSZ iofiltervpd[2098637]: IOFVPSSL_VerifySSLCertificate:238:Client certificate can't be verified


Environment

• vCenter Server 7.x
• vCenter Server 8.x

Cause

When the sms_self_signed certificate on the vCenter Server is missing on the ESXi host, the communication between the iofiltervpd service on the host and SPS (vmware-sps) service on the vCenter Server fails while verifying the certificate.

When the iofiltervpd service on the host connects to vCenter Server, it tries to verify the sms_self_signeds certificate stored within the vCenter's VECS SMS store against the certificate stored within the ESXi host file /etc/vmware/ssl/iofiltervp_castore.pem. This connection will fail if the sms_self_signed certificate is not available in the iofiltervp_castore.pem file on the host.

Resolution

Verification Steps

1. SSH to vCenter via root
2. List the sms_self_signed certificates
   
   /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store SMS --alias sms_self_signed
   
   
3. SSH to ESXi via root
4. View the /etc/vmware/ssl/iofiltervp_castore.pem file
   
   cat /etc/vmware/ssl/iofiltervp_castore.pem
   
   
5. Compare the certificate(s) listed in Step 2 against the pem file from Step 4. If the certificate from Step 2 is not available in the file from Step 4, continue with the resolution steps below.

Update the Certificate File

Manually copy the sms_self_signed certificate from vCenter Server to the ESXi host file /etc/vmware/ssl/iofiltervp_castore.pem on the host.

1. SSH to vCenter via root
2. Export and copy the certificate:
   
   /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store SMS --alias sms_self_signed
   
   Note: Copy the entire section including the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----".
   
   
3. SSH to ESXi via root
4. Edit the file /etc/vmware/ssl/iofiltervp_castore.pem
   
   vi /etc/vmware/ssl/iofiltervp_castore.pem
   
   
5. Paste the certificate copied from vCenter Server to this file (insert it at the beginning of the file)
6. Save and exit the vi editor
7. Restart the Iofiltervpd service on the host
   
   /etc/init.d/iofiltervpd restart
   
   
8. Wait for 2 minutes and the IOFilter status will show as Online on vCenter Server.

Additional Information

ESXi VASA Client Certificate Provision Has Failed