Workaround instructions to address CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105 in Tanzu Operations Manager

Products

Operations Manager

Issue/Introduction

CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105 has been determined to impact Tanzu Operation Manager via the Apache Log4j open source component it ships.

This vulnerability and its impact on VMware products are documented in the following VMware Security Advisory (VMSA). Review VMware Response to CVE-2021-44228: Apache Log4j Remote Code Execution (87068), CVE-2021-44228 – VMSA-2021-0028, and CVE-2021-45046 before continuing.

Environment

Product Version: 2.10

Resolution

Impact / Risks

Tanzu Operations Manager and UAA will be unavailable while it is being restarted on the Ops Manager VM. It will need to be unlocked via the web interface after the restart.

If the Tanzu Operations Manager VM is recreated, the resolution will need to be reapplied to it.

Resolution

Ops Manager patch is now available that use the Log4j 2.17 for CVE-2021-45105 (potential Denial of Service vulnerability caused by Log4j2):

This patches contains fixes for CVE-2021-44228 and CVE-2021-45046 as well as CVE-2021-45105.

Workaround

The workarounds described in this document are meant to be a temporary solution only.

To apply the workaround for CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105 to Tanzu Operations Manager VM perform the following steps:

1. SSH to the Tanzu Operations Manager VM by following these instructions: Logging Into Ops Manager VMs with SSH

2. Change to the root user:

sudo su

3. Download the patched CredHub release appropriate to your Ops Manager version onto the Ops Manager VM:

For Ops Manager 2.10, the CredHub release is at https://storage.googleapis.com/credhub-final-release-tarballs/credhub-2.9.8.tgz.
For Ops Manager 2.7 - 2.9, the CredHub release is at https://storage.googleapis.com/credhub-final-release-tarballs/credhub-2.5.16.tgz

Run this command to download the necessary patched CredHub release:

sudo -u tempest-web wget -O /var/tempest/internal_releases/credhub https://URL-FOR-APPROPRIATE-CREDHUB-RELEASE

4. Download the patched UAA release and apply it to the Ops Manager VM. Run the following commands:

For Ops Manager 2.8 - 2.10, the UAA release is at https://uaa-release-tarballs.s3.us-west-1.amazonaws.com/releases/uaa-release-74.5.30-rc.3.tgz
For Ops Manager 2.7, the UAA release is at https://uaa-release-tarballs.s3.us-west-1.amazonaws.com/releases/uaa-release-73.4.36-rc.2.tgz ( This release of UAA does not include a fix for CVE-2021-45105 )

sudo -u tempest-web wget -O /var/tempest/internal_releases/uaa https://URL-FOR-APPROPRIATE-UAA-RELEASE

5. Run the following commands to extract UAA release components on the Ops Manager VM:

sudo -u tempest-web bash -c "tar -Oxf /var/tempest/internal_releases/uaa packages/uaa.tgz | tar -Ozx ./uaa/cloudfoundry-identity-uaa.war > /home/tempest-web/uaa/tomcat/webapps/uaa.war"

sudo -u tempest-web bash -c "tar -Oxf /var/tempest/internal_releases/uaa packages/uaa.tgz | tar -Ozx ./uaa/cloudfoundry-identity-statsd.war > /home/tempest-web/uaa/tomcat/webapps/statsd.war"

6. Restart Tanzu Operations Manager:

service tempest-web restart

7. Apply Changes to the BOSH Director tile.

Verify workaround

BOSH Director

To verify the workaround for CVE-2021-44228, CVE-2021-45046, CVE-2021-45105 and has been correctly applied to the BOSH Director VM perform the following steps:

1. Open the "/debug/files" URL in the Ops Manager UI, such as https://pcf.mycorp/debug/files.

2. Click the link called "Bosh State".

3. Under the "releases" section verify the UAA release lists the appropriate version:

For Ops Manager 2.8-2.10, the uaa release should be "74.5.30-rc.3" as the version
For Ops Manager 2.7, the uaa release should be "73.4.36-rc.2" as the version ( This release of UAA does not include a fix for CVE-2021-45105 )

4. Under the "releases" section verify the credhub release lists the appropriate version:

For Ops Manager 2.10, the CredHub release should be version "2.9.8".
For Ops Manager 2.7 - 2.9, the CredHub release should be version "2.5.16".

Operations Manager VM

To verify the workaround for CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105 has been correctly applied to the Tanzu Operations Manager VM perform the following steps:

1. SSH onto the Ops Manager VM.

2. Change to the root user:

sudo su

3. Confirm that all log4j JAR files in the UAA directory have version 2.17:

ll -R /home/tempest-web/uaa | grep log4j

Revert workaround

To revert the workaround for CVE-2021-44228, CVE-2021-45046, CVE-2021-45105 to Tanzu Operations Manager perform the following steps:

1. Perform an export/import and recreate the Tanzu Operations Manager VM from a known good image.

2. Apply Changes to the BOSH Director tile.