Attempting offline bundle downloads using the LCM Bundle Transfer Utility fails with username or password is invalid for user
search cancel

Attempting offline bundle downloads using the LCM Bundle Transfer Utility fails with username or password is invalid for user

book

Article ID: 390340

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

  • When attempting to use the LCM Bundle Transfer Utility it fails with the following error:

Bundle Transfer Utility Tool failed with error : username or password is invalid for user
Caused by: com.vmware.evo.sddc.lcm.tools.bundletransfer.exception.BundleTransferUtilityException: Error occurred connecting to depot. Please try again.

Remedy: Please enter right username or password for user

  • The username/password being used for the LCM Bundle Transfer Utility can successfully authenticate to the support.broadcom.com portal
  • The username/password being used for the LCM Bundle Transfer Utility can successfully bring back the manifest files from the curl command below

curl -kv https://dl.broadcom.com:443/PROD2/evo/vmw/index.v3 -u depotusername

  • The LCM Debug logs show 'Network is unreachable' error messages:
YYYY-MM-DDT13:26:11.311-0700 [main] DEBUG [com.vmware.evo.sddc.lcm.tools.common.WarningsDisplayUtil]
                      Customer acknowledged to proceed.
YYYY-MM-DDT13:26:12.741-0700 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Getting file size for [/evo/vmw/index.v3] from URL[https://dl.broadcom.com:443/PROD2/evo/vmw/index.v3]
YYYY-MM-DDT13:26:12.749-0700 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.utils.CookieUtils]
                      VCF_DEPOT Depot Http Cookies: []
YYYY-MM-DDT13:26:12.937-0700 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Executing HEAD /PROD2/evo/vmw/index.v3
YYYY-MM-DDT13:26:14.064-0700 [main] ERROR [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Got exception while downloading manifest index [/evo/vmw/index.v3]: Network is unreachable
YYYY-MM-DDT13:26:14.110-0700 [main] DEBUG [com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility]
                      com.vmware.evo.sddc.lcm.tools.bundletransfer.exception.BundleTransferUtilityException: Error occurred connecting to depot. Please try again.
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityDownloader.validateUser(BundleTransferUtilityDownloader.java:195)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.validateDepotUser(BundleTransferUtility.java:2416)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.process(BundleTransferUtility.java:1157)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.parse(BundleTransferUtility.java:219)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.main(BundleTransferUtility.java:2361)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Caused by: com.vmware.evo.sddc.lcm.model.depot.exception.DepotConnectionFailureException: Internal error while validating credentials
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.validateUser(DepotDownloader.java:410)
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotBundleDownloadServiceImpl.validateUser(DepotBundleDownloadServiceImpl.java:262)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityDownloader.validateUser(BundleTransferUtilityDownloader.java:186)
        ... 12 more
Caused by: java.net.SocketException: Network is unreachable
        at java.base/sun.nio.ch.Net.connect0(Native Method)
        at java.base/sun.nio.ch.Net.connect(Unknown Source)
        at java.base/sun.nio.ch.Net.connect(Unknown Source)
        at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source)
        at java.base/java.net.SocksSocketImpl.connect(Unknown Source)
        at java.base/java.net.Socket.connect(Unknown Source)
        at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory$1.run(SSLConnectionSocketFactory.java:219)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:216)
        at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:148)
        at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:396)
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:158)
        at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:168)
        at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:136)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:190)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:133)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
        at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
        at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
        at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:75)
        at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:89)
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.getFileSize(DepotDownloader.java:132)
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.getFileSize(DepotDownloader.java:117)
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.getFileSize(DepotDownloader.java:334)
        at com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader.validateUser(DepotDownloader.java:346)
        ... 14 more
  • OR it will display the failure for SSL Handshake in OBTU log file:
YYYY-MM-DDT18:08:38.063+0530 [main] INFO  [com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility]
                      VMware Cloud Foundation LCM Bundle Transfer Tool, Version: 9.0.2.0.25151284
YYYY-MM-DDT18:08:38.064+0530 [main] INFO  [com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityDownloader]
                      Validating the depot user credentials...
YYYY-MM-DDT18:08:38.065+0530 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Downloading sourceFilePath /COMP/SDDC_MANAGER_VCF/index.v3 from host dl.broadcom.com port 443 and user dummy_download_token_user and isOfflineDepot false
YYYY-MM-DDT18:08:38.070+0530 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Getting file size for [/COMP/SDDC_MANAGER_VCF/index.v3] from URL[https://dl.broadcom.com:443/TOKEN/PROD/COMP/SDDC_MANAGER_VCF/index.v3]
YYYY-MM-DDT18:08:38.354+0530 [main] DEBUG [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Executing HEAD /TOKEN/PROD/COMP/SDDC_MANAGER_VCF/index.v3
YYYY-MM-DDT18:08:38.832+0530 [main] ERROR [com.vmware.evo.sddc.lcm.bundle.download.depot.DepotDownloader]
                      Got SSLHandshakeException connecting to dl.broadcom.com:443
YYYY-MM-DDT18:08:38.869+0530 [main] INFO  [com.vmware.vipclient.i18n.VIPCfg]
                      Formatting cache created.
YYYY-MM-DDT18:08:38.876+0530 [main] INFO  [com.vmware.vipclient.i18n.VIPCfg]
                      Translation Cache created.
YYYY-MM-DDT18:08:38.898+0530 [main] DEBUG [com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility]
                      com.vmware.evo.sddc.lcm.tools.bundletransfer.exception.BundleTransferUtilityException: username or password is invalid for user {0}
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityHelper.validateDepotUser(BundleTransferUtilityHelper.java:3140)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.process(BundleTransferUtility.java:873)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.parse(BundleTransferUtility.java:193)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtility.main(BundleTransferUtility.java:1843)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Caused by: com.vmware.evo.sddc.lcm.tools.bundletransfer.exception.BundleTransferUtilityException: Error occurred connecting to Dell depot. Please try again.
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityDownloader.validateUser(BundleTransferUtilityDownloader.java:202)
        at com.vmware.evo.sddc.lcm.tools.bundletransfer.BundleTransferUtilityHelper.validateDepotUser(BundleTransferUtilityHelper.java:3138)
        ... 11 more
Caused by: com.vmware.evo.sddc.lcm.model.depot.exception.DepotConnectionFailureException: Secure protocol communication error, check logs for more details
  • Running the following openssl command shows a different certificate chain than expected or will fail with an error instead of showing the certificate.

openssl s_client -showcerts -connect dl.broadcom.com:443

Expected output

CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = Palo Alto, O = Broadcom Inc., CN = hostupdate.vmware.com
verify return:1

Environment

VMware Cloud Foundation 5.1

VMware Cloud Foundation 5.2

Cause

This issue occurs due to SSL interception / a proxy server in use for HTTPS within the environment

 

Resolution

In order to successfully use the LCM Bundle Transfer Utility you will need to add the environment proxy certificates and the certificate for "dl.broadcom.com" to the Bundle Transfer Utility JRE default trust store that is returned from the openssl command above.

That can be accomplished using the steps below.

1. Identify the environment proxy certificates, if not known, use with the following command:

openssl s_client -showcerts -connect proxy_ip:443

If there is any Proxy Intermediate CA, need to add that certificate as well.

Take the certificate used to connect to Broadcom depot using below command:

openssl s_client -showcerts -connect dl.broadcom.com:443

2. Download all the certificates to the machine where the LCM Bundle Transfer Utility is being run.

3. Add each certificate to the Bundle Transfer Utility JRE default trust store with the following command. Modify the certificate file name to the certificate which you downloaded. 

/opt/obtu/jre/lin64/bin/keytool -importcert -file ca-bundle.crt -keystore /opt/obtu/jre/lin64/lib/security/cacerts

If you are using OBTU on a Windows machine, then follow the step below :

"OBTU_EXTRACTED_LOCATION\jre\win32\bin\keytool.exe" -importcert -alias ALIAS -file "PATH_TO_CERT_FILE.cer" -keystore "<OBTU_EXTRACTED_LOCATION\jre\win32\lib\security\cacerts"

NOTE - If prompted for a password use the default password of 'changeit'

4. Run the LCM Bundle Transfer Utility once again

Additional Information

Powered by Wolken Software