Cannot create new cluster from SDDC manager as it Fails to upload Image zip to vCenter

Products

• VMware SDDC Manager

Issue/Introduction

•  It is stuck on step - Upload vSphere Lifecycle Manager Image to vCenter Server.

/var/log/vmware/vcf/domainmanager/domainmanager.log:

[c.v.e.s.o.model.error.ErrorFactory,dm-exec-9]  [XXXX] VSPHERE_FAILED_TO_UPLOAD_ZIP_TO_VCENTER Failed to upload Image Zip [https://<sddcfqdn>/vmware/vcf/personalities/xxx/Content/OFFLINE_BUNDLE_xxxx.zip] to vcenter.
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to upload Image Zip [https://<sddc-fqdn>/vmware/vcf/personalities/xxx/Content/OFFLINE_BUNDLE_xxxx.zip] to vcenter.
        at com.vmware.vcf.common.contract.fsm.actions.impl.UploadZipToVcenterAction.execute(UploadZipToVcenterAction.java:79)

 

/var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server-log4cpp.log:

XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] Proxy-Connection: Keep-Alive
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188]
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] < HTTP/1.1 200 Connection established
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] <
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] * CONNECT phase completed
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] * CONNECT tunnel established, response 200
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] * ALPN: curl offers http/1.1
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] *  CAfile: /etc/pki/tls/certs/ca-bundle.crt
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] *  CApath: /etc/ssl/certs
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] * SSL certificate problem: unable to get local issuer certificate
XXXX-XX-XXTXX:XX:XX.XXXX verbose vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 188] * Closing connection
XXXX-XX-XXTXX:XX:XX.XXXX error vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 763] curl_easy_perform() failed: cURL Error: SSL peer certificate or SSH remote key was not OK, SSL certificate problem: unable to get local issuer certificate
XXXX-XX-XXTXX:XX:XX.XXXX error vmware-vum-server[XXX] [Originator@6876 sub=httpDownload] [httpDownloadPosix 763] [backtrace begin] product: VMware Update Manager, version: 8.0.3, build: build-24305161, tag: vmware-vum-server, cpu: x86_64, os: linux, buildType: release

Environment

VMware Cloud Foundation 4.x

VMware Cloud Foundation 5.x

Cause

vCenter Server is configured with a proxy which has SSL Inspection enabled. When SSL inspection is enabled, Proxy uses two connection for each request. 

• One between requester and proxy and the other between proxy and the actual https target.
• Requester on the connection to the proxy always sees the certificate from the proxy server instead of actual https target.
• Generally the proxy server certificates are either self signed or local CA signed.

Resolution

Configure the Proxy to disable SSL Inspection or add SDDC manager FQDN and IP to the No proxy list on the vCenter.

• Modify the /etc/sysconfig/proxy file with a vi editor and add the SDDC manager FQDN's and IP's to the following line, separated by a comma followed by a space character.

         For Example:        

 NO_PROXY="localhost, 127.0.0.1, <sddc-ip>, <sddc-manager-fqdn>"

• Apply the changes to the actual configuration using the following command:       

 #  export no_proxy="localhost, 127.0.0.1, <sddc-ip>, <sddc-manager-fqdn>"

• Note: Alternatively you can reboot the vCenter.
• Check the proxy settings are applied using the command:       

  #  env |grep -i proxy