Troubleshooting: Resolve VxRail Upgrade Failure when upgrading from VCF on VxRail 5.0.x to 5.1.x

Products

• VMware Cloud Foundation

Issue/Introduction

This article exists to guide users through the manual steps required for upgrading from VCF on VxRail 5.0.x to 5.1.x. Users are advised to follow the steps outlined in this KB article before initiating the upgrade process to ensure a smooth transition from VCF on VxRail 5.0.x to 5.1.x, and to avoid encountering the described error. Since the necessary steps are not available via the User Interface (UI) or API, this document serves as an essential resource to external users for minimizing downtime, avoiding potential conflicts, and ensuring that the upgrade is successful.

Symptoms:

VCF on VxRail Manager upgrade from 8.0.100 to 8.0.200/ 8.0.210 fails with the following error message on the task panel:
Note: Log excerpts are examples of 8.0.100 to 8.0.200, bundle id may change for different target versions, proceed with caution:

"Skipped VxRail cluster upgrade required patch binary:D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip is not available. File /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip does not exist."

The remediation message provided for this error recommends users to follow this KB to successfully upgrade their VxRail Manager Cluster from 8.0.100 to 8.0.2xx.
 

Environment

VMware Cloud Foundation 5.1.1
Vmware Cloud Foundation 5.1

Cause

The reason this issue occurred is that the VxRail 8.0.100 to 8.0.2xx upgrade requires an upgrade bundle that is not supported by VCF 5.1.x. The script and manual steps are intended to bridge this compatibility gap.

Resolution

Currently there is no resolution to the issue.

Workaround:

While the full automation of the upgrade process is being developed, the issue can be resolved by manually downloading the required bundle and checksum and executing the provided scripts to copy the artifacts onto the SDDC Manager. After that, on the SDDC Manager, modify the bundle and checksum permissions/owners by executing the script with another option.

1. Ensure the default VxRail bundle for VCF 5.1.x BoM is already downloaded or uploaded to the SDDC Manager.

2. Download the new upgrade bundle manually onto the machine with internet access from the Dell website -
   VCF 5.1 -  VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
   VCF 5.1.1 - VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip

3. Create a checksum file, similar as attached with this KB document.
   To create checksum file, follow below steps:
     i. Command: sha256sum <bundle-for-8.0.x.zip>
     ii. Create a new checksumfile and copy the output of Step 3(i) in it.

4. Download the script "scp_vxrail8x_bundles.sh" attached to this KB onto the same machine and ensure it has execute permission.

   1. Copy the bundle, checksum file and the script itself onto the SDDC Manager Appliance.
      Note: Examples below are for VxRail 8.0.200 bundle please change the variables for VxRail 8.0.x bundles.
      
      Expected File Name : Use the following value as per the table for VCF 5.1 and VCF 5.1.1

      Target VCF release	Expected File Name (--expected-file-name)
      VCF 5.1	D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
      VCF 5.1.1	HJ6NG_VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip

i. For Linux users, execute the scp_vxrail8x_bundles.sh script with "--transfer-bundles" and "--expected-file-name" option as shown below. Provide the filepath to the downloaded bundle and checksum and SDDC Manager IP with vcf user SSH credentials when prompted by the script.:

- Command :./scp_vxrail8x_bundles.sh --transfer-bundles --expected-file-name <bundle-file-name>
Example:./scp_vxrail8x_bundles.sh --transfer-bundles --expected-file-name VXRAIL_COMPOSITE-SLIM-8.0.200-28234746_for_8.0.x.zip
- Script Expected Output :
Enter the VxRail 8x bundle file&#39;s path: VxRail-8.0.200-Composite-
Upgrade-Slim-Package-for-8.0.x.zip
Enter the checksum file&#39;s path: D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum
Enter the SDDC Manager IP address: <SDDC_MANAGER_IP>
Copying K4G5K_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf@<SDDC_MANAGER_IP>:/nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
File VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
successfully copied to vcf@&lt;SDDC_MANAGER_IP&gt;:/nfs/vmware/vcf/nfs-
mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip
Copying D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip.checksum to vcf@&lt;SDDC_MANAGER_IP&gt;:/nfs/vmware/vcf/nfs-
mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip.checksum...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
File Checksum-for-VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.checksum successfully copied to vcf@<SDDC_MANAGER_IP>:/nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum
Copying script to vcf@<SDDC_MANAGER_IP>:/home/vcf/scp_vxrail8x_bundles.sh...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
Script successfully copied to vcf@<SDDC_MANAGER_IP>:/home/vcf/scp_vxrail8x_bundles.sh. For next steps, please refer to the Knowledge Base article at: https://kb.vmware.com/s/article/94747

ii. For windows users, use WinSCP to transfer the downloaded artifacts to the SDDC Manager appliance locations as below:

• Transfer the downloaded upgrade bundle to /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip

• Transfer the downloaded bundle checksum to /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum

• Transfer scp_vxrail8x_bundles.sh script to /home/vcf/scp_vxrail8x_bundles.sh

6. SSH to SDDC Manager, ssh vcf@<SDDC_MANAGER_IP>

7. Switch to root, su

8. Execute /home/vcf/scp_vxrail8x_bundles.sh with "--set-permissions" and "--expected-file-name" to ensure the artifacts have been correctly transferred and have correct permissions set.
    

   Expected File Name : Use the following value as per the table for VCF 5.1 and VCF 5.1.1

   Target VCF release	Expected File Name (--expected-file-name)
   VCF 5.1	D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
   VCF 5.1.1	HJ6NG_VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip

- Command :./scp_vxrail8x_bundles.sh --set-permissions --expected-file-name <bundle-file-name>
Example:./scp_vxrail8x_bundles.sh --transfer-bundles --expected-file-name VXRAIL_COMPOSITE-SLIM-8.0.200-28234746_for_8.0.x.zip
- Script Expected Output :
Setting permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to 755...
Successfully set permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to 755
Setting ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf_lcm:vcf...
Successfully set ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf_lcm:vcf
Setting permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to 755...
Successfully set permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to 755
Setting ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to vcf_lcm:vcf...
Successfully set ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to vcf_lcm:vcf

9. Log into the SDDC Manager UI to trigger the 5.1.x VxRail Manager upgrade.

Note: For VCF 5.1.1 vLCM based cluster upgrade will show precheck failure for this specific VxRail bundle. This can be safely ignored and upgrade can be continued.