Applying vRSLCM 2.1 patch 3, NSX-v 6.4.12, NSX-T 2.5.3.4.0, vCenter Server 6.7 Update 3q patch on VMware Cloud Foundation on 3.10.x release
search cancel

Applying vRSLCM 2.1 patch 3, NSX-v 6.4.12, NSX-T 2.5.3.4.0, vCenter Server 6.7 Update 3q patch on VMware Cloud Foundation on 3.10.x release

book

Article ID: 344791

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

The recommendation to customers on these prior VCF  3.x versions, is to upgrade to the latest VCF 3.11 release.
If customers are unable to do so, the purpose of this article is to provide guidance for such customers to upgrade just vRSLCM 2.1 patch 3, NSX-v 6.4.12, NSX-T 2.5.3.4.0, vCenter Server 6.7 Update 3q appliances

Affected Versions : 3.10.0, 3.10.1, 3.10.1.1, 3.10.1.2, 3.10.2, 3.10.2.1, 3.10.2.2

The information contained in this article applies to customers on both VCF on VxRail, and VCF (vSAN Ready Nodes)


Symptoms:

As documented in VMSA-2021-0028, all versions for the products prior to vRSLCM 2.1 patch 3, NSX-v 6.4.12, NSX-T 2.5.3.4.0, vCenter Server 6.7 Update 3q appliances are affected by the vulnerabilities listed in the advisory.

Since the VMware Cloud Foundation(VCF) 3.x versions, prior to VCF 3.11, bundle impacted releases of  vRSLCM, NSX-v, NSX-T, vCenter Server, the VCF versions 3.10.0, 3.10.1, 3.10.1.1, 3.10.1.2, 3.10.2, 3.10.2.1, 3.10.2.2 are similarly impacted by the vulnerabilities listed in the advisory


Environment

VMware Cloud Foundation 3.11

Cause

As documented in VMSA-2021-0028 all the VMware Cloud Foundation 3.x versions  prior to VCF 3.11 are affected by the vulnerabilities listed in the advisory

Resolution

VMware Cloud Foundation Version

Upgrade Options if Upgrading to VCF 3.11 is not possible 

Prior to VCF 3.10.0

Upgrade to 3.10.0 or later and follow the recommended approach below

VCF 3.10.x

Apply the steps in the Workaround section of this article



Workaround:


vRLSCM 2.1 Patch 3:

Follow the release notes to apply vRSLCM 2.1 patch 3
 

NSX-v 6.4.12:

Step 1: Perform below steps on each VMware NSX-v instance deployed in your VMware Cloud Foundation environment 

1) Apply the NSX-v 6.4.12 patch available at the Product Patch page to all NSX-v instances (Management & VI Domain) in the environment.


STEP 2: Perform below steps on each SDDC Manager VM deployed in your Cloud Foundation environment

1. Login to SDDC manager VM via SSH and sudo to root account

2. Verify the NSX-v version on the inventory 

root@sddc-manager [ /home/vcf ]# curl localhost/inventory/nsxmanagers | json_pp

"id" : "<<NSX-v ID>>",

"version" : "<<Current NSX-v Version>>",

"status" : "ACTIVE",

"hostName" : "nsxManager.vrack.vsphere.local",

"domainId" : "7d019faf-94fd-474c-9969-e4c4433c0ca2",

"managementIpAddress" : "10.0.0.9",

"vmName" : "nsxManager",

"vcenterId" : "72c116d5-b821-4b4b-84c7-8d0ac652b637"

Please note the following details

The field "id" in response, corresponds to <<NSX-v ID>>.
The "version" field for each of the NSX-v provides the <<Current NSX-v Version>>.

 

3. API to update NSX-v hot patch version: 6.4.12-19066632 on 3.10.x

root@sddc-manager [ /home/vcf ]# curl -X PATCH 'localhost/inventory/entities/<<NSX-v ID>>' -d '{"version":"6.4.12-19066632", "type":"NSXMANAGER"}' -H 'Content-Type:application/json'

 

4.  Verify the NSX-v Version

root@sddc-manager [ /home/vcf ]# curl localhost/inventory/nsxmanagers | json_pp

[

{

"vmName" : "nsxManager",

"domainId" : "7d019faf-94fd-474c-9969-e4c4433c0ca2",

"status" : "ACTIVE",

"hostName" : "nsxManager.vrack.vsphere.local",

"id" : "<<NSX-v ID>>",

"version" : "6.4.12-19066632",

...

"managementIpAddress" : "10.0.0.9",

"vcenterId" : "72c116d5-b821-4b4b-84c7-8d0ac652b637"

}

]

 

5. Update version-alias using below API  to support this async upgrade for future VCF compatible upgrades.

NOTE: This step needs to be done only once per SDDC Manager instance

curl 'http://localhost/v1/system/settings/version-aliases/NSX_MANAGER/<<Current NSX-v Version>>' -X PUT -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"aliases" : [ "6.4.12-19066632" ], "forceUpdate" : true}'

 

NSX-T 2.5.3.4:

Known Issue: After out of band upgrade to NSX-T 2.5.3.4, you will not be able to expand your VCF deployment, by adding NSX-T based Workload domains . To avoid such limitations, VMware recommends upgrading to VCF 3.11.
 

Step 1: Perform below steps on each VMware vCenter Server VM and each External PSC deployed in your VMware Cloud Foundation environment 

1) Apply the NSX-T 2.5.3.4.0 patch available at the Product Patch page to all NSX-T instances (VI Domain) in the environment.

 

STEP 2: Perform below steps on each SDDC Manager VM deployed in your Cloud Foundation environment

1. Login to SDDC manager VM via SSH and sudo to root account

2. API to update NSX-T hot patch version: 2.5.3.4.0-19069884 on 3.10.x

Get VCF deployed NSX-T Cluster Inventory Ids associated with domains.

root@sddc-manager [ ~ ]# curl localhost/inventory/nsxt| json_pp

[

{

"version" : "<<Current NSX-T Version>>",

"is_shared" : false,

"clusterIpAddress" : "10.0.0.154",

"clusterFqdn" : "vip-nsxmanager.vrack.vsphere.local",

"id" : "<<NSX-T ID>>",

"status" : "ACTIVE",

"nsxtClusterDetails" : [

{

"fqdn" : "nsxt-manager-1.vrack.vsphere.local",

"vmName" : "nsxt-manager-1",

"id" : "f7424394-f36d-4d46-b4fc-43f9cd10e8f2"

},

{

"vmName" : "vi-nsxmanager-1-2",

"fqdn" : "vi-nsxmanager-1-2.vrack.vsphere.local",

"id" : "5815c338-455e-4d7c-b21e-fc1f5b85b6b2"

},

{

"vmName" : "vi-nsxmanager-1-3",

"fqdn" : "vi-nsxmanager-1-3.vrack.vsphere.local",

"id" : "85ae537a-4cc5-4366-af25-31d699cc1f3f"

}

],

...

"domainIds" : [

"d82c8d79-6518-4e2d-a3b6-3bae168807c5"

]

}

]

Please note the following details

The field "id" in response, corresponds to <<NSX-T ID>> : NSX-T Cluster ID.
The "version" field for each of the NSX-v provides the <<Current NSX-T Version>.

 

3. For each NSX-T entity Id, update the NSX-T new version (Make sure you have already upgraded this NSX-T by logging to NSX-T cluster IP)

root@sddc-manager [ ~ ]# curl localhost/inventory/entities/<<NSX-T ID>> -X PATCH -d '{"type":"NSXT_CLUSTER","status":"ACTIVE","version":"2.5.3.4.0-19069884"}' -H 'Content-Type:application/json'

 

4.  Verify the NSX-T Version

root@sddc-manager [ ~ ]# curl localhost/inventory/nsxt| json_pp

[
{
"id" : "<<NSX-T ID>>",
"domainIds" : [
"d82c8d79-6518-4e2d-a3b6-3bae168807c5"
],
"status" : "ACTIVE",
"version" : "2.5.3.4.0-19069884",
"clusterFqdn" : "vip-nsxmanager.vrack.vsphere.local",
"is_shared" : false,
"nsxtClusterDetails" : [
{
"fqdn" : "nsxt-manager-1.vrack.vsphere.local",
"id" : "f7424394-f36d-4d46-b4fc-43f9cd10e8f2",
"vmName" : "nsxt-manager-1"
},
{
"vmName" : "vi-nsxmanager-1-2",
"fqdn" : "vi-nsxmanager-1-2.vrack.vsphere.local",
"id" : "5815c338-455e-4d7c-b21e-fc1f5b85b6b2"
},
{
"vmName" : "vi-nsxmanager-1-3",
"fqdn" : "vi-nsxmanager-1-3.vrack.vsphere.local",
"id" : "85ae537a-4cc5-4366-af25-31d699cc1f3f"
},

...
]
}
]

 

5. Update version-alias using below API to support this async upgrade for future VCF compatible upgrades. 

NOTE: This step needs to be done only once per SDDC Manager instance after it has been upgraded to VCF 3.11 via Skip level upgrade tool.

   a. Create version-alias.json JSON file in "/home/vcf" directory and add below content and save the file.

{
  "forceUpdate": true,
  "versionAliasesForBundleComponentTypes": [
    {
      "bundleComponentType": "NSX_T_MANAGER",
      "versionAliases": [
        {
          "version": "2.5.3.0.0-17558879",
          "aliases": [
            "2.5.3.4.0-19069884"
          ]
        }
      ]
     }
  ]
}

 

b. Execute the below API to update version-alias

# curl 'http://localhost/v1/system/settings/version-aliases' -k -X PUT -d @/home/vcf/version-alias.json -H 'Content-Type: application/json' -H 'Accept: application/json'

vCenter Server 6.7 u3q:

Step 1: Perform below steps on each VMware vCenter Server VM and each External PSC deployed in your VMware Cloud Foundation environment 

1) Powered off concurrent snapshots should be taken of all PSC's and VC's in the SSO domain prior to patching.

2) Apply the VMware vCenter server 6.7 Update 3q patch available at the Product Patch page to all external PSCs and vCenter Servers (Management & VI Domain) in the environment.


STEP 2: Perform below steps on each SDDC Manager VM deployed in your Cloud Foundation environment


1) Login to SDDC manager VM via SSH and sudo to root account

2)  Get PSC/VC ID from VCF inventory:
      To get vCenter/PSC details from VCF inventory run following command/Curl/API:

For vCenter Server

 

$ curl localhost/inventory/vcenters | json_pp

 

 Sample Output:
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  100 353 0 353 0 0 35300 0 --:--:-- --:--:-- --:--:-- 3530
  [
 {
 "hostName" : "vcenter-1.vrack.vsphere.local",
 "vmName" : "vcenter-1",
 "id" : "<vCenter_Id>",
 "version" : "<current version>",
"datastoreForVmDeploymentName" : "sfo01-m01-vsan",
"domainType" : "MANAGEMENT",
"status" : "ACTIVE",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"managementIpAddress" : "10.0.0.6"
}
]

For PSC

 

$ curl localhost/inventory/pscs | json_pp

 

Sample Output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 756 0 756 0 0 84000 0 --:--:-- --:--:-- --:--:-- 84000
[
{"domain" : "vsphere.local",
"bundleRepoDatastore" : "lcm-bundle-repo",
"status" : "ACTIVE",
"vmName" : "psc-2",
"hostName" : "psc-2.vrack.vsphere.local",
"id" : "<psc_Id>",
"replica" : true,
"version" : "<current version>",
"datastoreName" : "sfo01-m01-vsan",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"managementIpAddress" : "10.0.0.7",
"subDomain" : "vrack.vsphere.local"
},
{
"managementIpAddress" : "10.0.0.5",
"subDomain" : "vrack.vsphere.local",
"hostName" : "psc-1.vrack.vsphere.local",
"id" : "<psc_Id>",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domain" : "vsphere.local",
"status" : "ACTIVE",
"vmName" : "psc-1",
"datastoreName" : "sfo01-m01-vsan",
"version" : "<current version>",
"replica" : false,
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1"
}
]

 


The field "id" in response, corresponds to vCenter/PSC id.
The "version" field for each of the vCenter/PSC provides the current version of the vCenter/PSC.
 

3)  Update VCF inventory for vCenter Servers and PSCs


Note: Repeat below commands for all the vCenter Severs with their corresponding vcenter-id that were upgraded.

<SDDC_Manager_FQDN >  =  Fully qualified domain name of SDDC manager.
<vCenter_Id>  = Id of VCENTER for which version is to be updated in VCF inventory
<psc_Id>  = Id of PSC for which version is to be updated in VCF inventory
            
The build number of 6.7U3q is 19300125 so this is the version that needs to be inserted into the file i.e. 6.7.0-19300125

For vCenter Server

$ curl -X PATCH '<SDDC_Manager_FQDN >/inventory/entities/<vCenter_Id>' -d '{"version":"6.7.0-19300125", "type":"VCENTER"}' -H 'Content-Type:application/json'


For PSCs

$ curl -X PATCH '<SDDC_Manager_FQDN >/inventory/entities/<psc_Id>' -d '{"version":"6.7.0-19300125", "type":"PSC"}' -H 'Content-Type:application/json'

 

4) Verify vCenter Server and PSC versions

For vCenter Server

 

$ curl localhost/inventory/vcenters | json_pp

 

Sample Output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 353 0 353 0 0 35300 0 --:--:-- --:--:-- --:--:-- 35300
[
{
"hostName" : "vcenter-1.vrack.vsphere.local",
"vmName" : "vcenter-1",
"id" : "<vCenter_Id>",
"version" : "6.7.0-19300125",
"datastoreForVmDeploymentName" : "sfo01-m01-vsan",
"domainType" : "MANAGEMENT",
"status" : "ACTIVE",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"managementIpAddress" : "10.0.0.6"
}
]

 

 

For PSC

 

$ curl localhost/inventory/pscs | json_pp

 

Sample Output: % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 756 0 756 0 0 47250 0 --:--:-- --:--:-- --:--:-- 47250
[
{
"hostName" : "psc-2.vrack.vsphere.local",
"subDomain" : "vrack.vsphere.local",
"domain" : "vsphere.local",
"id" : "<psc_Id>",
"vmName" : "psc-2",
"version" : "6.7.0-19300125",
"datastoreName" : "sfo01-m01-vsan",
"bundleRepoDatastore" : "lcm-bundle-repo",
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"status" : "ACTIVE",
"managementIpAddress" : "10.0.0.7",
"replica" : true
},
{
"bundleRepoDatastore" : "lcm-bundle-repo",
"id" : "<psc_Id>",
"hostName" : "psc-1.vrack.vsphere.local",
"subDomain" : "vrack.vsphere.local",
"domain" : "vsphere.local",
"datastoreName" : "sfo01-m01-vsan",
"version" : "6.7.0-19300125",
"vmName" : "psc-1",
"managementIpAddress" : "10.0.0.5",
"replica" : false,
"domainId" : "68ae2add-db28-4671-9a92-f2a5b3dcaab1",
"status" : "ACTIVE"
}
]

 

 

  5)  Go to SDDCManager UI to verify the VC/PSC version after few mins


Note:

Repeat all relevant BOM patch for all domain in your Cloud Foundation environment.
Every time a new VI workload domain is created, these steps need to be performed.


Additional Information

Impact/Risks:

If the procedure documented below in the "Workaround" section is followed, the supported forward upgrade is VCF 3.11 using Skip level upgrade tool. Ensure you use latest skip level upgrade tool for VCF 3.x.

For information on the Skip level tool, please see:

VCF 3.11 SKIP level Upgrade tool
VCF 3.11 SKIP level Upgrade tool VCF on VxRail 3.11 SKIP level Upgrade tool