VxRail Upgrade: Dell Depot Connection Issues
search cancel

VxRail Upgrade: Dell Depot Connection Issues

book

Article ID: 404731

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

  • This article is meant to guide users through the manual steps required for both upgrading between major versions and updating betweenminor versions of VCF on VxRail, such as:
    • from VCF 4.x  to 5.x.y
    • or from VCF 5.0.x to 5.1.x/5.2.x (or 5.1.x to 5.2.x
  • Users are advised to follow the steps outlined in this article before initiating the upgrade process to ensure a smooth transition
  • Since the necessary steps are not available via the User Interface (UI) or API, this document serves as an essential resource to external users for minimizing downtime, avoiding potential conflicts, and ensuring that the upgrade is successful

NOTES:

Environment

  • VMware Cloud Foundation 4.x
  • VMware Cloud Foundation 5.x

Cause

SDDC Manager is unable to download VxRail bundles from Dell's depot because Dell has transitioned to an OAuth-based authentication system, which the current authentication mechanism within SDDC Manager is not compatible with. As a result, the connection to Dell's depot fails, preventing the download of VxRail bundles.

Resolution

  • Currently there is no resolution to the issue.
  • To work around this, follow the steps below:

Workaround Steps:

  1. Download the required VxRail Bundle from the Dell Website, to search in this website enter “Upgrade Slim Package for” in the keyword field and
    download the major bundle. 

    For example, to download the VxRail 8.0.210 Composite Upgrade Slim Package for 7.0.x see the screenshot:



  2. The downloaded bundle might end up with a file name different from the original one than in the download URL, for example, the download URL might be:

    https://dl.dell.com/downloads/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-7.0.x.zip

  3. But the file name after download ends up being:

    VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-7.0.x.zip

  4. Rename the file to have the same name as is used in the download URL, which in this example is:

    D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-7.0.x.zip

  5. Hover over the download button to get the URL with the right prefix, see the screenshot :



  6. Next, copy the VxRail bundle to the SDDC Manager Appliance into the folder /nfs/vmware/vcf/nfs-mount/bundle/depot/local/bundles

    Note: Due to security restrictions, the vcf user cannot directly SCP files into the final /nfs/vmware/vcf/nfs-mount/bundle/depot/local/bundles directory. As a workaround, SSH into the SDDC Manager, elevate to root (su -), and create a temporary folder in a directory with adequate storage (such as /nfs/vmware/vcf/nfs-mount/). Grant this temporary folder write permissions (chmod 777). You can then SCP the file into this temporary folder as the vcf user. Finally, use root privileges to move the file to its destination directory and apply the following chown and chmod settings.

  7. Then change the file permissions to 0755 using the chmod command. Going with the above example, you would run (the file name will differ for you, depending on which bundle is being used):
    chmod 0755 D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-7.0.x.zip
  8. Also change the file ownership of the bundle to vcf_lcm:vcf using this command:
    chown vcf_lcm:vcf D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-7.0.x.zip
  9. Upload VVS steps should be followed if the SDDC Manager version is 5.0 or higher

  10. Download the VVS file from the Dell Website (search for VVS and download the latest VVS file available), then upload the file via API call following one of the methods below:

      1.  If the SDDC Manager version is 5.2 or above, you can use the Developer center to initiate the API call:
        1. To do this, navigate to the Developer Center in the SDDC Manager UI
        2. then select the API Explorer
        3. In the API Explorer search for Compatibility Matrix similar to the screenshot below:
        4. For compatibilityMatrixSource enter VXRAIL_COMPAT
        5. For compatibilityMatrixContent select the VVS file
        6. Then click on [EXECUTE] to trigger the API call

      1. If the SDDC Manager version is between 5.0 or newer, but below 5.2 use either postman or CURL to upload the VVS file.
        1. When running the call within SDDC manager, the endpoint for the curl command will be localhost.
        2. Otherwise the endpoint with be the SDDC manager FQDN or IP address.
        3. The CURL command will look like this:
          curl --location --request PUT
'https://localhost/v1/compatibility-matrices?compatibilityMatrixSource=VXRAIL_COMPAT' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: Bearer <token>' \
--form 'compatibilityMatrixContent=@"/home/vcf/<vvsfile>.json"'
        4. Replace Token with the value obtained using the v1/tokens API as below.
          Token=$(curl -sk https://<sddc-manager>/v1/tokens -H 'Content-Type: application/json' -X POST -d '{ "username":"administrator@vsphere.local", "password":"<your_password>"}' | jq -r .accessToken)
          Notes:
          • /home/vcf/<vvsfile>.json will be the path of the VVS file, and needs to be changed as well, depending on where the VVS file is located and what its name is in the SDDC manager file system.
          • The VVS upload call above will fail to upload if there is no VMware VVS present. If you receive a failure, verify if the file VmwareCompatibilityData.json is present in /nfs/vmware/vcf/nfs-mount/compatibility
          • If you're using self-signed SDDC manager SSL certificates you will need to use the -k flag in the curl command to bypass the "self-signed certificate in certificate chain" error
  11. Next, upload the VxRail bundle that was downloaded at the start to the SDDC manager in order to make it available for the update. To do this, first download the specific Metadata JSON file depending on the SDDC version:

      1. For SDDC versions below 5.2, download the file from this URL (replace <broadcom_download_token> against your download token):
        https://dl.broadcom.com/<broadcom_download_token>/PROD/COMP/SDDC_MANAGER_VCF/partnerBundleMetadata.json
      2. For SDDC 5.2 or later versions, the file can be downloaded from the following URL (again replace <broadcom_download_token>):
        https://dl.broadcom.com/<broadcom_download_token>/PROD/COMP/SDDC_MANAGER_VCF/vxrailPartnerBundleMetadata.json
    •  
  12. Copy either partnerBundleMetadata.json or vxrailPartnerBundleMetadata.json to /home/vcf. Alternatively you can also use another folder, but do not use /nfs/vmware/vcf/nfs-mount/bundle).

  13. Change the file permissions for the file to 755 using the following command (replace /<path>/<to>/<file>/<partnerBundleMetadata_filename>):
    chmod 755 /<path>/<to>/<file>/<partnerBundleMetadata_filename>
  14. Change the file ownership for the file to vcf_lcm:vcf (again replace /<path>/<to>/<file>/<partnerBundleMetadata_filename>):
    chown vcf_lcm:vcf /<path>/<to>/<file>/<partnerBundleMetadata_filename>
  15. In the SDDC Manager UI, navigate to the Developer Center

  16. In the Developer Center go in the API explorer and search for the "Bundles" API as shown in the following screenshot


  17. Under POST method provide the following request body:
    {
 "partnerExtensionSpec": {
   "partnerBundleMetadataFilePath": "<partnerBundleMetadataPath>",
   "partnerBundleVersion": "8.0.300-28709350"
 }
}


  18. Replace <partnerBundleMetadataPath> with the path where the partnerBundleMetadata.json file was copied into in SDDC Manager. Also replace the string after partnerBundleVersion with the VxRail version identifier for the target version.

  19. If this process fails, check /var/log/vmware/vcf/lcm/lcm.log and look for the below error, else proceed to Step 20

    "can not read by LCM due to insufficient permissions, vcf_lcm user must have read and write access to /nfs/vmware/vcf/nfs-mount/bundle/depot/local/bundles directory..."

      • Check permission in /nfs/vmware/vcf/nfs-mount/bundle/depot/local/bundles, ensure vcf_lcm has read/write access to the bundle file
      • In case, permissions are configured properly, the actual bundle file name does not match what is listed in the metadata file. You can validate by running an ls against the absolute path of the bundle file shown in the error message. If the ls shows the file does not exist, rename the bundle file to the bundle name from the error message:
        mv path/to/bundle bundlename_from_error


  20. Once the upload was successful continue with the upgrade.

Additional Information

On certain situation if "/nfs/vmware/vcf/nfs-mount/bundle" is missing on SDDC manager, restart lcm service to get the directory auto created.

systemctl restart lcm

 

Powered by Wolken Software