CVE-2021-44228 & CVE-2021-45046: DX App Experience Analytics

Products

CA Application Experience Analytics SaaS (AXA)

Issue/Introduction

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228

Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints.

An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled.

From log4j 2.15.0, this behavior has been disabled by default. In previous releases (>2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class). Java 8u121 (see https://www.oracle.com/java/technologies/javase/8u121-relnotes.html) protects against remote code execution by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046

It was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. This could allows attackers with control over Thread Context Map (MDC) input data when the logging configuration uses a non-default Pattern Layout with either a Context Lookup (for example, $${ctx:loginId}) or a Thread Context Map pattern (%X, %mdc, or %MDC) to craft malicious input data using a JNDI Lookup pattern resulting in a denial of service (DOS) attack. Log4j 2.15.0 restricts JNDI LDAP lookups to localhost by default. Note that previous mitigations involving configuration such as to set the system property `log4j2.noFormatMsgLookup` to `true` do NOT mitigate this specific vulnerability. Log4j 2.16.0 fixes this issue by removing support for message lookup patterns and disabling JNDI functionality by default. This issue can be mitigated in prior releases (<2.16.0) by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class).

Environment

DX AXA SaaS

DX AXA 20.2.1, 21.3.1, 22.1

AXA 17.3.1, 17.3.2

Resolution

The DX AXA Engineering team has investigated and determined that few components of the App Experience Analytics contain the log4j versions that may be exposed to the vulnerability.

The Engineering team recommends the following steps to mitigate the exposure to this vulnerability.

For SaaS

The Engineering team has deployed a permanent fix on Dec 19.

For release 21.3.1

Engineering is actively testing a permanent fix on top of 21.3.1 with the Log4j 2.17.0 version. The customers will be notified accordingly.

The mitigation steps below are valid for CVE-2021-44228 (and not for CVE-2021-45046).

STEP#1: Add the environment variable “LOG4J_FORMAT_MSG_NO_LOOKUPS” with value “true” in axa-configmap and bpa-configmap as described below:

If you are using OpenShift

- Go to your dxi project

- Click Resources > Config Maps from left-navigation

- Select each of the config maps and click "Actions", then "Edit YAML"

- Add below line under the "data" section:

LOG4J_FORMAT_MSG_NO_LOOKUPS: "true"

- Click on "Save"

If you are using Kubernetes

kubectl edit configmaps axa-configmap -n<namespace>
kubectl edit configmaps bpa-configmap -n<namespace>

- Add the line:

LOG4J_FORMAT_MSG_NO_LOOKUPS: "true",' under "data"

Here is an example for Kubernetes:

- Save the file

STEP#2: Restart all Pods:

cd <DX_PLATFORM-HOME>tools

./dx-admin.sh stop

Wait for all pods to stop

./dx-admin.sh start

VERIFICATION:

If OpenShift:

Go to Application > Pods
Open axaservices-readserver
Click Terminal
Execute below command to verify new property has been added:
printenv | grep LOG4J_FORMAT_MSG_NO_LOOKUPS

If Kubernetes:

kubectl get pods -n <namespace> | grep axaservices-readserver
kubectl exec -ti <axaservices-readserver-podname> -n <namespace> sh
printenv | grep LOG4J_FORMAT_MSG_NO_LOOKUPS

Engineering is actively working on a permanent fix on top of 21.3.1 with the upgraded log4j2 version. The customers will be notified accordingly.

For release 20.2.1

There is no workaround and customers are recommended to upgrade to 21.3.1 and apply recommendations in above steps.

For releases 17.3.1 and 17.3.2

The Engineering team has provided mitigation steps for 17.3.1 and 17.3.2 on December 28th, 2021. The vulnerabilities are mitigated by removing the JndiLookup.class from jar file.

Pre-Requisites:

"zip" tool should be installed.

Remediation Steps

The below steps has to performed on all the nodes where the AXA compoents are running.

Stop all the running Services.
- Go to CA_EMM_HOME/bin directory and run ./stopServices.sh --all
Create a file "fixlog4shell.sh" in the CA_EMM_HOME directory with executable permissions and add the below content.
1. Go to CA_EMM_HOME directory
2. touch AXAlog4jFix.sh
3. chmod +x AXAlog4jFix.sh
4. Upate the file with below content:
  
  #!/bin/bash
  #set -x
  wget https://github.com/logpresso/CVE-2021-44228-Scanner/releases/download/v2.5.3/logpresso-log4j2-scan-2.5.3-linux.tar.gz
  tar -xf logpresso-log4j2-scan-2.5.3-linux.tar.gz
  chmod +x log4j2-scan
  echo y | ./log4j2-scan --fix ./ --backup-path $CA_EMM_HOME/log4jscan_backup.zip
Run the script AXAlog4jFix.sh with axauser
Add the below environment variable to bash profile.
- vi ~/.bashrc
- Append "export LOG4J_FORMAT_MSG_NO_LOOKUPS=true"
Start all the running Services.
- Go to CA_EMM_HOME/bin directory and run ./startServices.sh --all

Additional Information

https://support.broadcom.com/security-advisory/content/security-advisories/Broadcom-Enterprise-Software-Security-Advisory-for-Log4j-2-CVE-2021-44228-Vulnerability/ESDSA19792