Troubleshooting: Resolve VxRail Upgrade Failure when upgrading from VCF on VxRail 5.X to 5.Y

Products

• VMware Cloud Foundation

Issue/Introduction

This article exists to guide users through the manual steps required for upgrading from  VCF on VxRail 5.0.x to 5.1.x/5.2.x (or 5.1.x to 5.2.x) Users are advised to follow the steps  outlined in this KB article before initiating the upgrade process to ensure a smooth  transition from VCF on VxRail 5.0.x to 5.1.x/5.2.x (or 5.1.x to 5.2.x), and to avoid  encountering the described error. Since the necessary steps are not available via the  User Interface (UI) or API, this document serves as an essential resource to external  users for minimizing downtime, avoiding potential conflicts, and ensuring that the  upgrade is successful.

Symptoms: 

VCF on VxRail Manager upgrade from 8.0.1xx to 8.0.2xx/8.0.3xx (or 8.0.2xx to 8.0.3xx)  fails with the following error message on the task panel: Note: Log excerpts are  examples of 8.0.100 to 8.0.200, bundle id may change for different target versions,  proceed with caution: 

”Skipped VxRail cluster upgrade required patch binary:D1JM1_VxRail-8.0.200- Composite-Upgrade-Slim-Package-for-8.0.x.zip is not available. File  /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim Package-for-8.0.x.zip does not exist.” 

The remediation message provided for this error recommends users to follow this KB to  successfully upgrade their VxRail Manager Cluster from 8.0.100 to 8.0.2xx.

 

Environment

VMware Cloud Foundation 5.1
VMware Cloud Foundation 5.1.1
VMware Cloud Foundation 5.2

Cause

The reason this issue occurred is that the VxRail 8.0.100 to 8.0.2xx or 8.0.3xx upgrade requires an upgrade bundle that is not supported by VCF 5.1.x/VCF 5.2.x. The script and  manual steps are intended to bridge this compatibility gap. 

Resolution

Currently there is no resolution to the issue.

Workaround:

While the full automation of the upgrade process is being developed, the issue can be resolved by manually downloading the required bundle and checksum and executing the provided scripts to copy the artifacts onto the SDDC Manager. After that, on the SDDC Manager, modify the bundle and checksum permissions/owners by executing the script with another option.

1. Ensure the default target VxRail bundle for VCF 5.1.x/5.2.x BoM is already downloaded or uploaded to the SDDC Manager.

2. Download the new upgrade bundle manually onto the machine with internet access from the Dell website -

VCF 5.1 -  VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.1.1 - VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.2 - VxRail-8.0.300-Composite-Upgrade-Slim-Package-for-8.0.x.zip

3. Create a checksum file, similar as attached with this KB document :

To create checksum file, follow below steps:

  i. Command: sha256sum <bundle-for-8.0.x.zip>
  ii. Create a new checksumfile and copy the output of Step 3(i) in it.

 

4. Download the script "scp_vxrail8x_bundles.sh" attached to this KB onto the same machine and ensure it has execute permission.

1. Copy the bundle, checksum file and the script itself onto the SDDC Manager Appliance.
Note: Examples below are for VxRail 8.0.200 bundle please change the variables for VxRail 8.0.x bundles.

Expected File Name : Use the following value as per the table for VCF 5.1, VCF 5.1.1 and VCF 5.2

Target VCF release	Expected File Name (--expected-file-name)
VCF 5.1	D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.1.1	HJ6NG_VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.2	VPTWN_VxRail-8.0.300-Composite-Upgrade-Slim-Package-for-8.0.x.zip

i. For Linux users, execute the scp_vxrail8x_bundles.sh script with "--transfer-bundles" and "--expected-file-name" option as shown below. Provide the filepath to the downloaded bundle and checksum and SDDC Manager IP with vcf user SSH credentials when prompted by the script.:

- Command :./scp_vxrail8x_bundles.sh --transfer-bundles --expected-file-name <bundle-file-name>
Example:./scp_vxrail8x_bundles.sh --transfer-bundles --expected-file-name VXRAIL_COMPOSITE-SLIM-8.0.200-28234746_for_8.0.x.zip
- Script Expected Output :
Enter the VxRail 8x bundle file&#39;s path: VxRail-8.0.200-Composite-
Upgrade-Slim-Package-for-8.0.x.zip
Enter the checksum file&#39;s path: D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum
Enter the SDDC Manager IP address: <SDDC_MANAGER_IP>
Copying K4G5K_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf@<SDDC_MANAGER_IP>:/nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
File VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
successfully copied to vcf@&lt;SDDC_MANAGER_IP&gt;:/nfs/vmware/vcf/nfs-
mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip
Copying D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip.checksum to vcf@&lt;SDDC_MANAGER_IP&gt;:/nfs/vmware/vcf/nfs-
mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-
8.0.x.zip.checksum...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
File Checksum-for-VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.checksum successfully copied to vcf@<SDDC_MANAGER_IP>:/nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum
Copying script to vcf@<SDDC_MANAGER_IP>:/home/vcf/scp_vxrail8x_bundles.sh...
Warning: Permanently added '<SDDC_MANAGER_IP>' (ECDSA) to the list of known hosts.
Welcome to VMware Cloud Foundation Manager Virtual Appliance
Password:
Script successfully copied to vcf@<SDDC_MANAGER_IP>:/home/vcf/scp_vxrail8x_bundles.sh. For next steps, please refer to the Knowledge Base article at: https://kb.vmware.com/s/article/94747

ii. For windows users, use WinSCP to transfer the downloaded artifacts to the SDDC Manager appliance locations as below:

• Transfer the downloaded upgrade bundle to /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip

• Transfer the downloaded bundle checksum to /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum

• Transfer scp_vxrail8x_bundles.sh script to /home/vcf/scp_vxrail8x_bundles.sh

 

5. SSH to SDDC Manager, ssh vcf@<SDDC_MANAGER_IP>

6. Switch to root, su

7. Execute /home/vcf/scp_vxrail8x_bundles.sh with "--set-permissions" and "--expected-file-name" to ensure the artifacts have been correctly transferred and have correct permissions set.

 

Expected File Name : Use the following value as per the table for VCF 5.1, VCF 5.1.1 and 5.2.

Target VCF release	Expected File Name (--expected-file-name)
VCF 5.1	D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.1.1	HJ6NG_VxRail-8.0.210-Composite-Upgrade-Slim-Package-for-8.0.x.zip
VCF 5.2	VPTWN_VxRail-8.0.300-Composite-Upgrade-Slim-Package-for-8.0.x.zip

- Command :./scp_vxrail8x_bundles.sh --set-permissions --expected-file-name <bundle-file-name>
Example:./scp_vxrail8x_bundles.sh --set-permissions --expected-file-name D1JM1_VxRail-8.0.200- Composite-Upgrade-Slim-Package-for-8.0.x.zip
- Script Expected Output :
Setting permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to 755...
Successfully set permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to 755
Setting ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf_lcm:vcf...
Successfully set ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip to vcf_lcm:vcf
Setting permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to 755...
Successfully set permissions for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to 755
Setting ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to vcf_lcm:vcf...
Successfully set ownership for /nfs/vmware/vcf/nfs-mount/bundle/D1JM1_VxRail-8.0.200-Composite-Upgrade-Slim-Package-for-8.0.x.zip.checksum to vcf_lcm:vcf

8. Log into the SDDC Manager UI to trigger the 5.1.x/5.2.x VxRail Manager upgrade.

Note: For VCF 5.1.1 vLCM based cluster upgrade will show precheck failure for this specific VxRail bundle. This can be safely ignored and upgrade can be continued.