Unable to remediate ESXi hosts from a image based cluster. Fails with error: Failed to create DRS maintenance mode requests. This is a prerequisite for putting hosts into maintenance. Reason: 'Reached maximum allowed retry attempts'

Products

• VMware vSphere ESXi
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

Error: Failed to create DRS maintenance mode requests. This is a prerequisite for putting hosts into maintenance. Reason: 'Reached maximum allowed retry attempts'

/var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log

YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1858787] [Originator@6876 sub=StsClient opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [ssoUtils 90] Failed to get Saml token after refreshing STS certs error: SSL Exception: Verification parameters:
YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1858787] [Originator@6876 sub=SamlAsyncApiProvider opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [SamlAsyncApiProvider 77] Failed to add Saml auth provider error:SSL Exception: Verification parameters:^M
YYYY-MM-DDTHH:MM:SS.Z warning vmware-vum-server[1858787] [Originator@6876 sub=ClusterOps opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [ClusterOps 874] Failed to create maintenance requests. Reason: Error: Error:
YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1865884] [Originator@6876 sub=IO.Http opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] User agent failed to send request; (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:^M
YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1858787] [Originator@6876 sub=ClusterOps opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [ClusterOps 882] Failed to invoke requests. Reason: Reached maximum allowed retry attempts. Giving Up
YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1858787] [Originator@6876 sub=RemediateClusterTask opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] Failed to create maintenance requests. Reached maximum allowed retry attempts. Giving Up^

 

YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1858787] [Originator@6876 sub=StsClient opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [ssoUtils 90] Failed to get Saml token after refreshing
 STS certs error: SSL Exception: Verification parameters:
--> PeerThumbprint: 2D:##:12:##:97:##:44:#3:8C:##:49:##:7B:##:7B:##:81:##:CA:##
--> ExpectedThumbprint:
--> ExpectedPeerName: Hostname
--> The remote host certificate has these problems:
-->
--> * certificate has expired
YYYY-MM-DDTHH:MM:SS.Z warning vmware-vum-server[1858787] [Originator@6876 sub=ClusterOps opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] [ClusterOps 874] Failed to create maintenance reques
ts. Reason: Error: Error:
-->    com.vmware.vapi.std.errors.unauthenticated
--> No messages!
--> . Retry after 10 seconds
YYYY-MM-DDTHH:MM:SS.Z error vmware-vum-server[1865884] [Originator@6876 sub=IO.Http opID=f9c6cc96-e4eb-411a-8d01-4f410e5d2e7c] User agent failed to send request; (null), N7Vmacore3Ssl1
8SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: 2D:##:12:##:97:##:44:#3:8C:##:49:##:7B:##:7B:##:81:##:CA:##
--> ExpectedThumbprint:
--> ExpectedPeerName:Hostname 
--> The remote host certificate has these problems:
-->
--> * certificate has expired)
--> [context]zKq7AVECAQAAAA8jcwEOdm13YXJlLXZ1bS1zZXJ2ZXIAADMcU2xpYnZtYWNvcmUuc28AAP4XQgAfP0MAjJlKAGwbQQChi0EAyB9BAKkmQQC+QUEABOw3ABdFOAC7D1EBsI4AbGlicHRocmVhZC5zby4wAALf+g9saWJjLnNvLjYA[/
context]

 

Cause

The issue may be caused if the SAML token exchange and SSL handshake is failing between vCenter and ESXi hosts. This can occur for a few reasons, such as a time mismatch between the vCenter and ESXi hosts, or issues with the STS certificates.

 

Resolution

• Verify if there is any time difference between vCenter and ESXi hosts by validating the NTP configuration.
• Renew STS certificates and re-try the remediate.
• Check Machine SSL Certificate(Missing CA).

 

Additional Information

Checking Expiration of STS Certificate on vCenter Servers
"Signing certificate is not valid"