ASync BOM patching procedure to patch ESXi 7.0.U1E for VCF 4.1.x and VCF 4.2.x, and 7.0.U2E for 4.3.x environments

Products

VMware Cloud Foundation

Issue/Introduction

VMSA-2022-0004 details numerous vulnerabilities in VMware ESXi.
As part of the response, VMware released "back in time" patches for VMware ESXi 7.0U1 and VMware ESX 7.0U2.
This KB documents the process should customers running VMware Cloud Foundation 4.1.x, 4.2.x or 4.3.x wish to update their ESXi hosts to these releases.

Release Notes for VMware ESXi 7.0U1e
Release Notes for VMware ESXi 7.0U2e

NOTE: This KB is not applicable for VCF on Dell EMC VxRail Release

Symptoms:
As documented in VMSA-2022-0004 , all versions of the VMware ESXi 7.0 prior to 7.0 Update 3c are affected by the vulnerabilities listed in the advisory.
Since the VMware Cloud Foundation(VCF) 4.x versions, prior to VCF 4.4, bundle impacted releases of VMware ESXi, the VCF versions VCF 4.1.x, VCF 4.2.x and VCF 4.3.x releases, are similarly impacted by the vulnerabilities listed in the advisory.

Environment

VMware Cloud Foundation 4.3.x
VMware Cloud Foundation 4.2.x
VMware Cloud Foundation 4.1

Resolution

Guidance Steps:

Verify the VCF release to be able to apply ESXi Async patch.

VCF Release	Upgrade Steps
VCF 4.1.0	Upgrade to VCF 4.1.0.1, Apply KB 87050 and then move to Step 2.
VCF 4.1.0.1	Apply KB 87050 and then move to Step 2.
VCF 4.2.0	Upgrade to VCF 4.2.1, Apply KB 87050 and then apply KB 88287.
VCF 4.2.1	Apply KB 87050 and then apply KB 88287.
VCF 4.3.0	Upgrade to VCF 4.3.1.1 as documented in VCF 4.3.1.1 Release Notes and apply KB 88287.
VCF 4.3.1	Upgrade to VCF 4.3.1.1 as documented in VCF 4.3.1.1 Release Notes and apply KB 88287.
VCF 4.3.1.1	Apply KB 88287.

Apply ESXi fix patch async outside of VCF.

Release Notes:
For VCF 4.1.x and 4.2.x, refer VMware ESXi 7.0 Update 1e
For VCF 4.3.x, refer VMware ESXi 7.0 Update 2e

Using vSphere product documentation apply ESXi patch directly without using SDDC Manager.

Note: After ESXi hosts are async patched. Please make sure to cleanup the baselines and ISOs on vSphere UI which were created as part of the upgrade through vSphere. If not deleted, the future upgrades from VCF LCM are prone to failures.
Update the SDDC Manager Inventory.

Following steps can be used to update SDDC Manager inventory:
3.1 Download the tar file from attachments.
3.2 SCP the file to SDDC Manager in "/tmp" directory.
3.3 SSH to SDDC Manager using "vcf" user and switch to root using "su" command.
3.4 On the SDDC manager, cd to "/tmp" directory and extract the tar file.

Extract the tar file on the SDDC Manager
root@sddc-manager [~]# cd /tmp
root@sddc-manager [/tmp]# tar xvf lcm-tools-prod.tar.gz

3.5 The executable script can be found in the bin directory.

usage: bin/inventory-sync <SDDC SSO USER> <SDDC SSH USER>
Once the above command is executed with the appropriate parameters, the script will prompt as below. Please provide the credentials when prompted.

Enter SDDC Manager SSH Password:
Enter SDDC Manager Root User's Password:
Enter SDDC Manager SSO Password:

Below is a sample run:
---
vcf@sddc-manager [ /tmp ]$ bin/inventory-sync [email protected] vcf
2022-02-18 16:50:19.130 [WARN ] Cloud Sleuth not configured.
2022-02-18 16:50:20.254 [INFO ] VCF Async Patch Tool - Version: 4.4.0-vcf4400RELEASE-19311902
2022-02-18 16:50:20.308 [INFO ] Log file is generated at /home/vcf/tmp/bin/async_patch_tool.log
Enter SDDC Manager SSH Password:
Enter SDDC Manager Root User's Password:
Enter SDDC Manager SSO Password:
2022-02-18 16:50:41.284 [INFO ] Performing global inventory sync.
2022-02-18 16:50:41.284 [INFO ] Validating inventory sync input spec
2022-02-18 16:50:41.308 [INFO ] Performing inventory sync for entities [VCENTER, ESXI, NSXT_CLUSTER]
2022-02-18 16:50:41.308 [INFO ] Constructing version diff for vcenters
2022-02-18 16:50:42.767 [INFO ] Retrieved 1 vcenter(s) from the inventory.
2022-02-18 16:50:43.311 [INFO ] Creating new session on vcenter-1.vrack.vsphere.local for [email protected]
2022-02-18 16:50:43.821 [INFO ] Get the current appliance system version
---

3.6 After the inventory-sync finishes executing, the SDDC manager inventory will be in sync with all the hosts that were updated.

Steps to enable future upgrades from VCF (one time activity per SDDC manager instance).

Patch Applied	Recommended Upgrade to ESXi Version	Corresponding VCF Version
7.0Update 1e	7.0 Update 3c	VCF 4.4
7.0Update 2e	7.0 Update 3c	VCF 4.4

For offline customers, upload the VCF 4.4 patch bundles following VCF 4.4 release documentation.

4.1 Get access token:

Login to SDDC Manager via SSH and run following command

curl '<SDDC Manager FQDN>/v1/tokens' -i -X POST -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"username" : "<SSO User ID>","password" : "<SSO Password>"}'

<SDDC Manager FQDN > - Fully qualified domain name of SDDC manager.
<SSO User ID> - SSO user id of SDDC VM
<SSO Password> - SSO password for SDDC VM

Example :

Request:
curl 'http://sddc-manager.vrack.vsphere.local/v1/tokens' -i -X POST -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"username" : "[email protected]","password" : "password@123"}'

Response:{"accessToken":"eyJhbG...bWluaXN","refreshToken":{"id":"8702601b-cc1d-4ed2-acc4-8ccf8f849999"}}

4.2 Enable forward upgrades by following below steps based on VCF Version

Steps for VCF 4.1.x

Steps for VCF 4.2.x

Steps for VCF 4.3.x

Forward upgrade path for customers on VCF 4.1.x and VCF 4.2.x:

To enable forward upgrades, please follow below steps:

Create the API request payload:

Open a vi editor with file name request.json

vi request.json

Create a file request.json with below contents:

{
"forceUpdate": true,
"versionAliasesForBundleComponentTypes": [ {
"bundleComponentType": "ESX_HOST",
"versionAliases": [ {
"aliases": ["7.0.1-19324898"],
"version": "7.0.2-18426014"
} ]
} ]
}

Trigger the PUT api call using the request.json payload and the output for the API call is as below.

API: curl -k '<SDDC Manager FQDN>/v1/system/settings/version-aliases' -X PUT -d @request.json -H 'Content-Type:application/json' -H 'Authorization: Bearer <AUTH TOKEN>'

<SDDC Manager FQDN > - Fully qualified domain name of SDDC manager.
<AUTH TOKEN> - Access Token retrieved from Step 4.1.

Sample Output:

{
"elements": [{
"bundleComponentType": "ESX_HOST",
"versionAliases": [{
"version": "7.0.2-18426014",
"aliases": [
"7.0.1-19324898"
]
}]
}
]
}

Forward upgrade path for customers on VCF 4.3.x:

To enable forward upgrades, please follow below steps:

Create the API request payload:

Open a vi editor with file name request.json

vi request.json

Create a file request.json with below contents:

{
"forceUpdate": true,
"versionAliasesForBundleComponentTypes": [ {
"bundleComponentType": "ESX_HOST",
"versionAliases": [ {
"aliases": ["7.0.2-19290878"],
"version": "7.0.2-18426014"
} ]
} ]
}

Trigger the PUT api call using the request.json payload and the output for the API call is as below.

API: curl -k '<SDDC Manager FQDN>/v1/system/settings/version-aliases' -X PUT -d @request.json -H 'Content-Type:application/json' -H 'Authorization: Bearer <AUTH TOKEN>'

<SDDC Manager FQDN > - Fully qualified domain name of SDDC manager.
<AUTH TOKEN> - Access Token retrieved from Step 4.1.

Sample Output:
{
"elements": [{
"bundleComponentType": "ESX_HOST",
"versionAliases": [{
"version": "7.0.2-18426014",
"aliases": [
"7.0.2-19290878"
]
}]
}
]
}

Note:
VI domain creation will still be based on the VCF release BOM. For example: VI domain creation on VCF 4.2.1 will be based on the ESXi version(7.0.1-17551050) i.e 4.2.1 release BOM. Follow the Guidance Steps above to patch ESXi for a new VI.

Attachments

lcm-tools-prod.tar get_app