Brownfield convert task fails during "Installing SDDC Manager VCSA certificate" phase.
search cancel

Brownfield convert task fails during "Installing SDDC Manager VCSA certificate" phase.

book

Article ID: 406578

calendar_today

Updated On:

Products

VMware SDDC Manager

Issue/Introduction

  • Brownfield convert or import task fails during "Installing SDDC Manager VCSA certificate" phase with following error:



  • Logs on SDDC Manager (/var/log/vmware/vcf/domainmanager/domainmanager.log) will show similar to below snippets:

YYYY-MM-DDThh:mm:ss INFO  [vcf_dm,#######################,f600] [c.v.e.s.s.InstallSddcManagerVmcaCertificateLocalAction,dm-exec-13]  Installing SDDC Manager VCSA certificate
YYYY-MM-DDThh:mm:ss ERROR [vcf_dm,#######################,f600] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-13]  [F88SM4] SDDC_MANAGER_INSTALL_CERT_FAILED Failed to install VMCA Certificate on SDDC Manager <SDDC-FQDN>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to install VMCA Certificate on SDDC Manager <SDDC-FQDN>
.
.
YYYY-MM-DDThh:mm:ss DEBUG [vcf_dm,#######################,f600] [c.v.e.s.o.c.ProcessingTaskSubscriber,dm-exec-13]  Collected the following errors for task with name InstallSddcManagerVmcaCertificateLocalAction and ID 7f000001-xxxx-xxxx-xxxx-836c9bbd000f: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.evo.sddc.sddcmanager.messages)]]

  • Logs on SDDC Manager (/var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log) will show similar to below snippets:

YYYY-MM-DDThh:mm:ss INFO  [common,#######################,71ad] [c.v.e.s.a.u.utils.SslCertValidator,http-nio-127.0.0.1-7100-exec-9] Resolving dns name : <SDDC-FQDN>, to match with ip list ; [<SDDC-IP>]
YYYY-MM-DDThh:mm:ss INFO  [common,#######################,71ad] [c.v.e.s.a.u.utils.DnsResolutionUtils,http-nio-127.0.0.1-7100-exec-9] Dns name <SDDC-FQDN>, resolved to IPs [127.0.0.1]
YYYY-MM-DDThh:mm:ss ERROR [common,#######################,71ad] [c.v.e.s.a.u.utils.SslCertValidator,http-nio-127.0.0.1-7100-exec-9] Certificate validations failed
java.security.cert.CertificateException: Hostname in CN field [<SDDC-FQDN>] could not be resolved to an IP address of the SDDC manager [<SDDC-IP>]
.
.
YYYY-MM-DDThh:mm:ss ERROR [common,#######################,71ad] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7100-exec-9] [OF0KS8] CERT_REPLACEMENT_FAILED Cannot replace existing certificate with the input cert. Validations did not pass.
Make sure the input cert chain is valid. The structure must be:
server cert followed by intermediate certs followed by CA cert
OR
A self signed server cert
All certs in the chain must conform to X.509 standards.
Also make sure that the DNS name in both the CN field and the optional Subject Alternative Name extension, is a resolvable hostname
com.vmware.evo.sddc.appliance.utilities.error.ApplianceManagerException: Cannot replace existing certificate with the input cert. Validations did not pass.
Make sure the input cert chain is valid. The structure must be:
server cert followed by intermediate certs followed by CA cert
OR
A self signed server cert
All certs in the chain must conform to X.509 standards.
Also make sure that the DNS name in both the CN field and the optional Subject Alternative Name extension, is a resolvable hostname

  • Logs on SDDC Manager (/var/log/messages) will show similar to below snippets:

YYYY-MM-DDThh:mm:ss <SDDC-Short-Name> vmbase_init[1090]: + sh -c 'echo "127.0.0.1   <SDDC-FQDN>" >> /etc/hosts'

Environment

VCF 5.2.x.x

VCF 9.0

Cause

  • During SDDC Manager cert replacement/installation, the commonsvcs api is being called which does a validation of fqdn resolution which comes as an input with the certificate.
  • This validation is failing because recently the dns java library got upgraded from 2.1.9 to 3.6.3. This can cause DNS issues due to significant API changes and incompatibility between versions.

Resolution

Take a snapshot of the SDDC manager VM and follow the steps below:

  1. SSH to SDDC Manager with vcf user and su to root.

  2. Back up the /etc/hosts file:
      cp /etc/hosts /etc/hosts.bak

  3. Edit the /etc/hosts file:
      vi /etc/hosts

  4. Comment out the line where the SDDC FQDN is mapped to 127.0.0.1 (add # at the start of the line).

127.0.0.1   localhost.localdomain
127.0.0.1   localhost
127.0.0.1   photon# End /etc/hosts (network card version)
127.0.0.1   FQDN_of_SDDC manager

     5. Retry brownfield convert/import operation. 

     6. Once the operation completes successfully, revert the changes to /etc/hosts file.

Note: 

SDDC Manager patch update fails during PostgreSQL service startup due to inconsistencies in /etc/hosts file.

Health-check operation SDDC fails with DNS-check | Forward DNS lookup is NOT configured as expected

Additional Information

Powered by Wolken Software