Workaround instructions to address CVE-2021-44228 in Tanzu Observability Proxy

Products

• Observability

Issue/Introduction

CVE-2021-44228 has been determined to impact Tanzu Observability Proxy via the
Apache Log4j open source component it ships.

This vulnerability and its impact on VMware products are documented in the following VMware Security Advisory (VMSA), please review this document before continuing:

• CVE-2021-44228 – VMSA-2021-0028

Resolution

Impact / Risks

There is no expected impact from performing the workaround described. 

Resolution

NOTE: Tanzu Observability Proxy version 10.12 is available for download. This version addresses CVE-2021-44228. The workaround described in this document is not necessary with version 10.12 or above.

The workarounds described in this document are meant to be a temporary solution only.

Upgrades documented in the aforementioned advisory should be applied to remediate CVE-2021-44228 when available.

Workaround

To apply the workaround for CVE-2021-44228 to the Tanzu Observability pProxy, perform the following steps:

1. Ensure that Tanzu Observability proxy is version 5.x or above. Check the version of the Tanzu Observability Proxy,

a. Log in to the Tanzu Observability UI.

b. Select Browse > Proxies.

c. Note the Tanzu Observability Proxy version number in the versions column.

If you have to upgrade, refer to Install and Manage Wavefront Proxies.

2. When starting the Java virtual machine, set the log4j2.formatMsgNoLookups parameter to True, as follows:

Docker

Add “-Dlog4j2.formatMsgNoLookups=true” to the “$JAVA_ARGS” environment
variable.

For example:

docker run -d \
-e WAVEFRONT_URL=https://<myinstance>.wavefront.com/api/ \
-e WAVEFRONT_TOKEN=<YOUR_API_TOKEN> \
-e JAVA_ARGS=“-Dlog4j2.formatMsgNoLookups=true” \
-p 2878:2878 projects.registry.vmware.com/tanzu_observability/proxy:latest

Kubernetes

Add “-Dlog4j2.formatMsgNoLookups=true” to the “$JAVA_ARGS” environment
variable.

Manual install: If you installed manually, you can add the variable to your proxy deployment. See https://github.com/wavefrontHQ/wavefront-collector-for-kubernetes/blob/ad7ec8493e4[...]822f0084c3d8bf18122845/deploy/kubernetes/6-wavefront-proxy.yam.

Helm install: If you installed via helm, run helm repo update to get the latest chart definitions, and then upgrade your change.

Then run helm upgrade --namespace <your_namespace><your_chartname> wavefront/wavefront.

By default, your namespace and chartname are both wavefront.

Linux

Edit your “/etc/sysconfig/wavefront-proxy” file to add the “$JAVA_ARGS” environment variable. Example:

Export JAVA_ARGS=“-Dlog4j2.formatMsgNoLookups=true”

Mac

Edit your ”/usr/local/opt/wfproxy/bin/wfproxy” file to add the “$JAVA_ARGS” environment variable.

For example:

Export JAVA_ARGS=“-Dlog4j2.formatMsgNoLookups=true”

Windows

Install the latest version of the proxy, which has the workaround (but NOT the
final remediation.

Verify workaround

To verify the workaround for CVE-2021-44228 has been correctly applied to

Tanzu Observability proxy, follow these steps:

1. Ensure that Tanzu Tanzu Observability Proxy is version 5.x or above.
To check the version of the Tanzu Observability Proxy,

a. Log in to the Tanzu Observability Proxy UI.  

b. Select Browse > Proxies.

c. Note the Tanzu Observability Proxy version number in the versions column. The
version must be 5.x or above.

2. Connect to your proxy platform (or Docker image) and run the following:

wavefront [ / ]$ ps -ef | grep wavefront-proxy

wavefront       12     1 10 10:15:58 ?    00:00:08 java -

XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=85.0 -Dlog4j2.formatMsgNoLookups=true -

Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -

Dlog4j.configurationFile=/etc/wavefront/wavefront-proxy/log4j2.xml -jar /opt/wavefront/wavefront-
proxy/bin/wavefront-proxy.jar -h https://<hostname>.wavefront.com/api/ -t <token> --hostname 
b81be395e671 --ephemeral true --buffer /var/spool/wavefront-proxy/buffer --flushThreads 6

 

Revert workaround

To revert the workaround for CVE-2021-44228 to Tanzu Observability Proxy follow these steps:

WARNING: Do not revert unless you have upgraded to Tanzu Observability Proxy v10.11 or above.

1. Remove “-Dlog4j2.formatMsgNoLookups=true” from the “$JAVA_ARGS” environment
variable or the Wavefront configuration file when starting the Java Virtual Machine.

Verify workaround removal

To verify the workaround for CVE-2021-44228 has been correctly removed from the
Tanzu Observability proxy perform the following steps:

1. To check the version of the Tanzu Observability Proxy, use the Tanzu Observability UI and navigate to Browse Proxies. Note the Tanzu Observability Proxy version number in the versions column. The version should be 10.11 or above.

2. Connect to your Proxy platform (or Docker image) and run “ps -ef | grep wavefront-proxy”.
log4j2.formatMsgNoLookups should no longer be in the output;

wavefront [ / ]$ ps -ef | grep wavefront-proxy

wavefront       12     1 10 10:15:58 ?    00:00:08 java -XX:InitialRAMPercentage=50.0 -
XX:MaxRAMPercentage=85.0 -Dlog4j2.formatMsgNoLookups=true -

Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -

Dlog4j.configurationFile=/etc/wavefront/wavefront-proxy/log4j2.xml -jar /opt/wavefront/wavefront-
proxy/bin/wavefront-proxy.jar -h https://<hostname>.wavefront.com/api/ -t <token> --hostname
 b81be395e671 --ephemeral true --buffer /var/spool/wavefront-proxy/buffer --flushThreads 6