Removal of a non-responsive ESXi Host(s) fail at task "Remove vmknics(s) from ESXi Hosts" to create resource warning

Products

• VMware Cloud Foundation 5.x

Issue/Introduction

Symptoms:
During the removal of a non-responsive ESXi Host from a cluster, the task "Remove vmknics(s) from ESXi Hosts" fails with an error message "Failed to remove vmknics from host ..." having a cause like "Failed to create warning for resource with ID <an UUID here> javax.net.ssl.SSLPeerUnverifiedException: Hostname <the SDDC Manager hostname here> not verified".

 

The exception in the /var/log/vmware/vcf/domainmanager/domainmanager.log would look like:

yyyy-mm-ddThh:mm:ss ERROR [vcf_dm,66795c0c47e57cda2491a372c44f0a9f,4933] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-20]  [R98EQL] VSPHERE_REMOVE_VMKNICS_FAILED Failed to remove vmknics from host <ESXi Hostname here>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to remove vmknics from host <ESXi Hostname here>
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:142)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:42)
        ...
Caused by: com.vmware.evo.sddc.common.core.error.InvalidStateException: null
        at com.vmware.vcf.common.util.WarningUtil.addWarning(WarningUtil.java:155)
        at com.vmware.vcf.common.util.WarningUtil.addWarning(WarningUtil.java:105)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:119)
        ... 16 common frames omitted
Caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException: javax.net.ssl.SSLPeerUnverifiedException: Hostname <the SDDC Manager hostname here> not verified:
    certificate: <certificate details>
    DN: CN=<the SDDC Manager hostname here>, OU=<certificate details>, O=<certificate details>, L=<certificate details>, ST=<certificate details>, C=<certificate details>
    subjectAltNames: []
        at com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiClient.execute(ApiClient.java:711)
        at com.vmware.cloud.foundation.rest.commonsvcs.service.ResourceWarningsApi.createResourceWarningWithHttpInfo(ResourceWarningsApi.java:142)
        at com.vmware.cloud.foundation.rest.commonsvcs.service.ResourceWarningsApi.createResourceWarning(ResourceWarningsApi.java:128)
        at com.vmware.vcf.common.util.WarningUtil

 

It is important to note that in the cause of "javax.net.ssl.SSLPeerUnverifiedException" in the above example log, the "subjectAltNames" is an empty list.

 

 

Environment

VMware Cloud Foundation 5.1.x and forward

Cause

When trying to create a resource warning upon the deletion of vmknics from the ESXi Hosts, an API request will be made from the domain manager service to commonsvcs. This API request is using HTTPS and has the SDDC Manager hostname as the base path for the API request. 

Resolution

This will be fixed for 5.2.1 and following releases.

Workaround:

1. Replace only the SDDC Manager certificate using the SDDC Manager UI or API.
2. Replacing the certificate with ensure the subject alternative name(s) list will be populated at least with the common name.
3. Once the SDDC Manager certificate is replaced, retry the failed remove unresponsive host operation. 

Additional Information

Managing Certificates in VMware Cloud Foundation