Unable to Connect to Online Depot and errors out: "Internal error while validating the credentials. Network is unreachable."

Products

• VMware SDDC Manager

Issue/Introduction

When trying to connect to Online Depot 9.x getting error "Internal error while validating the credentials. Network is unreachable."

/var/log/vmware/vcf/lcm/lcm-debug.log:

yyyy-dd-mmT18:48:59.141+0000 INFO [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.v.l.r.a.c.v.s.DepotSettingsController, http-nio-127.0.0.1-7400-exec-7] Update Depot Settings: {"vmwareAccount":{"downloadToken":"XXX"}}
yyyy-dd-mmT18:48:59.142+0000 INFO [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.v.l.s.i.DepotSettingsServiceImpl, http-nio-127.0.0.1-7400-exec-7] validating VCF_DEPOT account
yyyy-dd-mmT18:48:59.142+0000 DEBUG [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.e.s.l.b.d.depot.DepotDownloader, http-nio-127.0.0.1-7400-exec-7] Downloading sourceFilePath /metadata/productVersionCatalog/v1/productVersionCatalog.json from host dl.broadcom.com port 443 and user dummy_download_token_user and isOfflineDepot false
yyyy-dd-mmT18:48:59.142+0000 DEBUG [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.e.s.l.b.d.depot.DepotDownloader, http-nio-127.0.0.1-7400-exec-7] Getting file size for [/metadata/productVersionCatalog/v1/productVersionCatalog.json] from URL[https://dl.broadcom.com:443/XXX/PROD/metadata/productVersionCatalog/v1/productVersionCatalog.json]
yyyy-dd-mmT18:48:59.166+0000 DEBUG [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.e.s.l.b.d.depot.DepotDownloader, http-nio-127.0.0.1-7400-exec-7] Executing HEAD /XXX/PROD/metadata/productVersionCatalog/v1/productVersionCatalog.json
yyyy-dd-mmT18:48:59.283+0000 INFO [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [o.a.h.c.h.i.c.HttpRequestRetryExec, http-nio-127.0.0.1-7400-exec-7] Recoverable I/O exception (java.net.SocketException) caught when processing request to {s}->https://dl.broadcom.com:443
yyyy-dd-mmT18:48:59.391+0000 ERROR [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.e.s.l.b.d.depot.DepotDownloader, http-nio-127.0.0.1-7400-exec-7] Got exception while downloading file [/metadata/productVersionCatalog/v1/productVersionCatalog.json]: Network is unreachable
yyyy-dd-mmT18:48:59.392+0000 ERROR [vcf_lcm, 663e6c1b805bde6b3fcb759a391890fc, 9975] [c.v.v.l.r.a.c.v.s.DepotSettingsController, http-nio-127.0.0.1-7400-exec-7] Update Depot Settings com.vmware.evo.sddc.lcm.model.depot.exception.DepotConnectionFailureException: Internal error while validating credentials at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.validateUser(DepotDownloader.java:566)

Validated the connectivity running below command:

curl -v --head https://dl.broadcom.com:443/<Token>/PROD/COMP/SDDC_MANAGER_VCF/index.v3

Output of above command:

* Host dl.broadcom.com:443 was resolved. 
* IPv6: XXX 
* IPv4: XXX 
.
* Immediate connect fail for XXX: Network is unreachable 
* Trying XXX :443... 
* ALPN: curl offers http/1.1 
* TLSv1.3 (OUT), TLS handshake, Client hello (1): 
* CAfile: /etc/pki/tls/certs/ca-bundle.crt 
* CApath: none
 * TLSv1.3 (IN), TLS handshake, Server hello (2): 
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): 
* TLSv1.3 (IN), TLS handshake, Certificate (11): 
* TLSv1.3 (OUT), TLS alert, bad certificate (554): 
* SSL certificate problem: certificate is not yet valid 
* closing connection #0 curl: (60) SSL certificate problem: certificate is not yet valid More details here: https://curl.se/docs/sslcerts.html

 

Environment

VCF 9.x

Cause

The curl: (60) SSL certificate problem causes due to a time synchronization discrepancy and the VCF builder was not synchronized with the authoritative NTP (Network Time Protocol) server.

Henceforth the system clock was set incorrectly (behind the certificate's "Not Before" date), the SSL handshake failed as the system perceived the Broadcom certificate to be not yet valid. This triggered the "Network is unreachable" error within the vcf-lcm logs.

 

Resolution

Add the valid NTP server details and then try to connect to Online depot.