Applying vCenter Server 6.7 Update 3n patch on VMware Cloud Foundation on 3.9.x, 3.10.0, 3.10.1.x, 3.10.2

Products

VMware Cloud Foundation

Issue/Introduction

The purpose of this article is to provide guidance for affected customers using VMware Cloud Foundation versions 3.9.0, 3.9.1, 3.10.0, 3.10.1, 3.10.1.1, 3.10.1.2, and 3.10.2.

Symptoms:
vCenter Server 6.7 versions (embedded or external PSC) prior to 6.7 U3n is affected by CVE-2021-21985 and CVE-2021-21986. As described in the VMware security advisory.

Similarly as described in the advisory, VMware Cloud Foundation(VCF) 3.x versions are affected by CVE-2021-21985 and CVE-2021-21986.

VMSA-2021-0020 published on September 21st 2021 documents vulnerabilities in VMware vCenter Server versions prior to 6.7U3o, which also impact VCF3.x versions.
Please see KB 85719 for further details

Environment

VMware Cloud Foundation 3.10.1.2
VMware Cloud Foundation on VxRail 3.9.x
VMware Cloud Foundation 3.9.x
VMware Cloud Foundation 3.10.1.1
VMware Cloud Foundation on VxRail 3.10.x
VMware Cloud Foundation 3.10.x

Resolution

To resolve this issue for VMware Cloud Foundation 3.9.0, 3.9.1, 3.10.0, 3.10.1, 3.10.1.1, 3.10.1.2, and 3.10.2, upgrade to version 3.10.2.1.

If you are unable to upgrade at this time, apply the steps in the Workaround section of this article.

Workaround:
Notes:

For VMware Cloud Foundation 3.10.2, you must upgrade to 3.10.2.1 to resolve this issue.
If you are using VMware Cloud Foundation versions earlier to VCF 3.9 are required to first upgrade to version 3.9 or later before following the workaround steps below.
For more information on this vulnerability, refer to the advisory VMSA-2021-0010 and vCenter 6.7u3n available in VMware vCenter server 6.7 Update 3n Release Notes.
Take a snapshot of the PSC and vCenter Server before applying the patch.

Steps to follow:

Apply the VMware vCenter server 6.7 Update 3n patch available at the Product Patch page to all external PSCs and vCenter Servers (Management & VI Domain) in the environment.
Update VCF inventory following the steps below:
1. Login to SDDC manager VM via SSH.
2. Get PSC/VC ID from VCF inventory:

To get vCenter/PSC details from VCF inventory run following command/Curl/API:

$ curl localhost/inventory/vcenters | json_pp

Sample Output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 353 0 353 0 0 35300 0 --:--:-- --:--:-- --:--:-- 35300
[
{
"hostName" : "vcenter-1.vrack.vsphere.local",
"vmName" : "vcenter-1",
"id" : "<vCenter/psc_Id>",
"version" : "<current version>",
"datastoreForVmDeploymentName" : "sfo01-m01-vsan",
"domainType" : "MANAGEMENT",
"status" : "ACTIVE",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"managementIpAddress" : "10.0.0.6"
}
]

$ curl localhost/inventory/pscs | json_pp

Sample Output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 756 0 756 0 0 84000 0 --:--:-- --:--:-- --:--:-- 84000
[
{
"domain" : "vsphere.local",
"bundleRepoDatastore" : "lcm-bundle-repo",
"status" : "ACTIVE",
"vmName" : "psc-2",
"hostName" : "psc-2.vrack.vsphere.local",
"id" : "<vCenter/psc_Id>",
"replica" : true,
"version" : "<current version>",
"datastoreName" : "sfo01-m01-vsan",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"managementIpAddress" : "10.0.0.7",
"subDomain" : "vrack.vsphere.local"
},
{
"managementIpAddress" : "10.0.0.5",
"subDomain" : "vrack.vsphere.local",
"hostName" : "psc-1.vrack.vsphere.local",
"id" : "<vCenter/psc_Id>",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domain" : "vsphere.local",
"status" : "ACTIVE",
"vmName" : "psc-1",
"datastoreName" : "sfo01-m01-vsan",
"version" : "<current version>",
"replica" : false,
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1"
}
]
The field "id" in response, corresponds to vCenter/PSC id.
The "version" field for each of the vCenter/PSC provides the current version of the vCenter/PSC.

Update VCF inventory for vCenter Servers and PSCs

Note: Repeat below commands for all the vCenter Severs with their corresponding vcenter-id that were upgraded.
<SDDC_Manager_FQDN > - Fully qualified domain name of SDDC manager.
<vCenter/psc_Id> - Id of VCENTER/PSC for which version is to be updated in VCF inventory
6.7.0-18010531 - Version of vCenter/PSC patch that was applied on hosts.

For vCenter Server

$ curl -X PATCH '<SDDC_Manager_FQDN >/inventory/entities/<vCenter/psc_Id>' -d '{"version":"6.7.0-18010531", "type":"VCENTER"}' -H 'Content-Type:application/json'

For PSCs

$ curl -X PATCH '<SDDC_Manager_FQDN >/inventory/entities/<vCenter/psc_Id>' -d '{"version":"6.7.0-18010531", "type":"PSC"}' -H 'Content-Type:application/json'

Verify vCenter Server and PSC versions

$ curl localhost/inventory/vcenters | json_pp

Sample Output: % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 756 0 756 0 0 47250 0 --:--:-- --:--:-- --:--:-- 47250
[
{
"hostName" : "psc-2.vrack.vsphere.local",
"subDomain" : "vrack.vsphere.local",
"domain" : "vsphere.local",
"id" : "<vCenter/psc_Id>",
"vmName" : "psc-2",
"version" : "6.7.0-18010531",
"datastoreName" : "sfo01-m01-vsan",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"status" : "ACTIVE",
"managementIpAddress" : "10.0.0.7",
"replica" : true
},
{
"bundleRepoDatastore" : "lcm-bundle-repo",
"id" : "<vCenter/psc_Id>",
"hostName" : "psc-1.vrack.vsphere.local",
"subDomain" : "vrack.vsphere.local",
"domain" : "vsphere.local",
"datastoreName" : "sfo01-m01-vsan",
"version" : "6.7.0-18010531",
"vmName" : "psc-1",
"managementIpAddress" : "10.0.0.5",
"replica" : false,
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"status" : "ACTIVE"
}
]

Go to SDDCManager UI to verify the VC version after few mins

Note: Make sure when a new Workload domain is created, apply all the steps mentioned above in 1 and 2.

Additional Information

Applying vCenter Server 7.0 update patch to address vulnerabilities in VMSA-2021-0010 on VMware Cloud Foundation 4.1, 4.1.0.1, 4.2 (84271)

Impact/Risks:

After applying the vCenter Server 6.7u3n patch on your VCF environment using procedure below, supported forward upgrade is VCF 3.10.1.2 using Skip level upgrade tool.
Ensure you use latest skip level upgrade tool for VCF 3.x. For VCF on VXrail you may encounter a known issue during upgrade to VCF 3.10.1.2 , Follow the KB guidance https://kb.vmware.com/s/article/84457
VCF 3.10.1.2 SKIP level Upgrade tool
VCF on VxRail 3.10.1.2 SKIP level Upgrade tool