HA configuration is stuck at the "Election" state and doesn't proceed further for a set of hosts.
Article ID: 345414
Updated On:
Products
VMware vCenter Server
VMware vSphere ESXi
Issue/Introduction
Symptoms:
HA configuration is stuck at the "Election" state and doesn't proceed further for a set of hosts.
The fdm logs of the master host will report Untrusted thumbprint errors.
2023-09-20T01:36:46.548Z info fdm[180268] [Originator@6876 sub=Cluster opID=SWI-71792bc4] Untrusted thumbprint (30:8D:63:95:DE:4A:3D:09:9F:33:19:56:DE:B1:DC:19:1F:F1:3A:63) for host (192.168.0.81)- failing verify 2023-09-20T01:36:46.548Z verbose fdm[180268] [Originator@6876 sub=Cluster opID=SWI-71792bc4] Blacklisting ip address 192.168.0.81 for 60 seconds 2023-09-20T01:36:46.548Z verbose fdm[180268] [Originator@6876 sub=Cluster opID=SWI-71792bc4] IP 192.168.0.81 marked bad for reason Invalid Credentials 2023-09-20T01:36:46.549Z warning fdm[180268] [Originator@6876 sub=Cluster opID=SWI-71792bc4] Failed to verify host (192.168.0.81) - closing connection 2023-09-20T01:36:46.550Z verbose fdm[180268] [Originator@6876 sub=Message opID=SWI-71792bc4] Accept completion callback error N5Vmomi5Fault13SecurityError9ExceptionE(Fault cause: vmodl.fault.SecurityError --> ) --> [context]zKq7AVECAQAAAP/bJAESZmRtAACoS+ZmZG0AAKMj3gDlB9IA5pTUABmSawCaEW4AybpvAGLtbwB47m8Azn5+AGOEfgBb/HsA9fx7AA0Q2wBdYdsAvPrYATt9AGxpYnB0aHJlYWQuc28uMAACvacObGliYy5zby42AA==[/context] 2023-09-20T01:36:46.550Z info fdm[180268] [Originator@6876 sub=Message opID=SWI-71792bc4] Destroying connection 2023-09-20T01:36:48.543Z info fdm[180371] [Originator@6876 sub=Cluster opID=SWI-598dc61d] Untrusted thumbprint (9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:FD) for host (192.168.0.82)- failing verify 2023-09-20T01:36:48.544Z verbose fdm[180371] [Originator@6876 sub=Cluster opID=SWI-598dc61d] Blacklisting ip address 192.168.0.82 for 60 seconds 2023-09-20T01:36:48.544Z verbose fdm[180371] [Originator@6876 sub=Cluster opID=SWI-598dc61d] IP 192.168.0.82 marked bad for reason Invalid Credentials 2023-09-20T01:36:48.544Z warning fdm[180371] [Originator@6876 sub=Cluster opID=SWI-598dc61d] Failed to verify host (192.168.0.82) - closing connection 2023-09-20T01:36:48.544Z verbose fdm[180371] [Originator@6876 sub=Message opID=SWI-598dc61d] Accept completion callback error N5Vmomi5Fault13SecurityError9ExceptionE(Fault cause: vmodl.fault.SecurityError --> ) --> [context]zKq7AVECAQAAAP/bJAESZmRtAACoS+ZmZG0AAKMj3gDlB9IA5pTUABmSawCaEW4AybpvAGLtbwB47m8Azn5+AGOEfgBb/HsA9fx7AA0Q2wBdYdsAvPrYATt9AGxpYnB0aHJlYWQuc28uMAACvacObGliYy5zby42AA==[/context] 2023-09-20T01:36:48.544Z info fdm[180371] [Originator@6876 sub=Message opID=SWI-598dc61d] Destroying connection
SSL Thumbprint in VCDB for the impacted hosts:
root@vc1 [ ~ ]# psql -U postgres -d VCDB -c "select id,dns_name,ip_address,host_ssl_thumbprint,expected_ssl_thumbprint from vpx_host;" id | dns_name | ip_address | host_ssl_thumbprint | expected_ssl_thumbprint ----+-------------------+--------------+-------------------------------------------------------------+------------------------------------------------------------- 30 | esxi1.gsslabs.org | 192.168.0.81 | 9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:EE | 9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:EE 24 | esxi2.gsslabs.org | 192.168.0.82 | 9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:FF | 9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:FF 27 | esxi3.gsslabs.org | 192.168.0.83 | 54:D1:FC:45:43:9E:31:8D:DE:EA:11:4F:84:01:4C:08:0D:F6:2A:12 | 54:D1:FC:45:43:9E:31:8D:DE:EA:11:4F:84:01:4C:08:0D:F6:2A:12 (3 rows)
SSL Thumbprint of the current certificate installed in the hosts:
Impacted hosts: [root@esxi1:~] openssl x509 -in /etc/vmware/ssl/rui.crt -text -fingerprint |grep -i fingerprint SHA1 Fingerprint=30:8D:63:95:DE:4A:3D:09:9F:33:19:56:DE:B1:DC:19:1F:F1:3A:63 [root@esxi2:~] openssl x509 -in /etc/vmware/ssl/rui.crt -text -fingerprint |grep -i fingerprint SHA1 Fingerprint=9C:E4:F5:A4:9A:A6:8D:26:94:57:2E:F6:70:ED:03:5D:14:14:BE:FD Working host: [root@esxi3:~] openssl x509 -in /etc/vmware/ssl/rui.crt -text -fingerprint |grep -i fingerprint SHA1 Fingerprint=54:D1:FC:45:43:9E:31:8D:DE:EA:11:4F:84:01:4C:08:0D:F6:2A:12
Environment
VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x
VMware vSphere ESXi 7.x
VMware vCenter Server 8.0.x
VMware vSphere ESXi 7.x
Cause
This can occur when there is a mismatch of thumbprints between VCDB and the actual host's SSL certificate.
Such a mismatch could occur after the SSL certificate of the host is updated with custom certificates but not got synced with VCDB.
Such a mismatch could occur after the SSL certificate of the host is updated with custom certificates but not got synced with VCDB.
Resolution
Disconnect and reconnect the impacted hosts to update the host's current SSL thumbprint in VCDB.
(Rebooting the host or restart of the services wouldn't update the VCDB).
Additional Information
Impact/Risks:
HA cluster will not be formed.
HA cluster will not be formed.
Feedback
Yes
No
Powered by
