Expired ESXi host certificates can impact vSAN functionality
search cancel

Expired ESXi host certificates can impact vSAN functionality

book

Article ID: 326667

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

Expired ESXi host certificate(s) in a vSAN cluster can have negative impacts to vSAN functionality such as:

  • incomplete unicast agent list on hosts resulting in unhealthy/inaccessible objects
  • esxcli vsan commands failing
  • The primary node not receiving performance data from other hosts in the cluster
  • vCenter/ESXi communication
In the /var/run/log/hostd.log file in the ESXi host, you see entries similar to:

<YYYY-MM-DD>T<TIME>Z error hostd[B182B70] [Originator@6876 sub=Default opID=378619de-c4-3001 user=vpxuser:com.vmware.vsan.health] AdapterServer caught exception: SSL Exception: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired

<YYYY-MM-DD>T<TIME>Z info hostd[31240B70] [Originator@6876 sub=VsanSimsStubImpl opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Need to retry fetchVsanSharedSecret. Count 3, Period 10, loginSeq 11303
...
<YYYY-MM-DD>T<TIME>Z info hostd[31281B70] [Originator@6876 sub=VsanSimsStubImpl opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Need to retry fetchVsanSharedSecret. Count 2, Period 10, loginSeq 11304
...
<YYYY-MM-DD>T<TIME>Z info hostd[312C2B70] [Originator@6876 sub=VsanSimsStubImpl opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Need to retry fetchVsanSharedSecret. Count 1, Period 10, loginSeq 11305
...
<YYYY-MM-DD>T<TIME>Z info hostd[31281B70] [Originator@6876 sub=VsanSimsStubImpl opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Need to retry fetchVsanSharedSecret. Count 0, Period 10, loginSeq 11306
<YYYY-MM-DD>T<TIME>Z info hostd[31281B70] [Originator@6876 sub=VsanSimsStubImpl opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Invoke fetchVsanSharedSecret failed for last time
<YYYY-MM-DD>T<TIME>Z error hostd[31A44B70] [Originator@6876 sub=Default opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] AdapterServer caught exception: SSL Exception: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired

<YYYY-MM-DD>T<TIME>Z error hostd[31A44B70] [Originator@6876 sub=Default opID=88a25ce2-edc4-11eb-70-af-7033 user=vpxuser:com.vmware.vsan.health] Backtrace:

In the /var/log/vsanvpd.log file in the ESXi host, you see entries similar to:

<YYYY-MM-DD>T<TIME>Z vsanSoapServer: run:182:Failed to accept client <IP Address> [30]: SSL_ERROR_SSL error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown
<YYYY-MM-DD>T<TIME>Z vsanSoapServer: run:186:SOAP process done
<YYYY-MM-DD>T<TIME>Z vsanSoapServer: run:139:To accept SOAP socket

 

In the /var/log/vmware/vsan-health/vmware-vsan-health-service.log located in the vCenter Server, you see entries similar to:

<YYYY-MM-DD>T<TIME>Z INFO vsan-health[sq1368:t2] [VsanMgmtAdapters::_HandleOneHost] Member info for host host-10(<ESXi hostname>) is (vim.cluster.VsanPerfMemberInfo) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   thumbprint = '65374cbd9fe51889014158b834b6ef7be56e0fa7',
   memberUuid = u'host-10;62e7855e-a9e9-d339-3642-0050569b1ce8',
   isSupportUnicast = true,
   unicastAddressInfos = (vim.cluster.VsanUnicastAddressInfo) []
}


Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Environment

VMware vSAN 6.5.x

Cause

Expired ESXi host certificates impede proper communication among hosts in a vSAN cluster

Resolution

At the first sign of ESXi host certs expiring or about to expire renew the certs by right-clicking the host > Certificates > Renew Certificate

Workaround:



Additional Information

Powered by Wolken Software