Layer 7 APIM Product Suite vulnerability summary for Log4J Zero-Day: (CVE-2021-44228, CVE-2021-45046, CVE-2021-4104, CVE-2019-17571, CVE-2021-44832)
Article ID: 230205
Updated On:
Products
Issue/Introduction
A zero-day vulnerability within a widely used java logging library called log4j has been detected.
Environment
Gateway: 9.4, 10.0, 10.1
Portal: 4.4, 4.5, 5.x
Live API Creator: 5.4
Resolution
This KB article will cover log4j in relation to all Layer7 products. It will be updated frequently as any new information is determined. Please feel free to check back early and often. Opening a support case for status will get this link provided as the current state.
We recommend reading this article from the API Academy on preventing passthrough of log4j threat via gateway policy.
=================
Layer7 API Gateway 10.x log4j Removal notice in upcoming version(s).
=================
We have cleaned up by deleting offending classes from remaining Log4j 1.x occurrences in 3rd party libraries.
We have released a SSG platform patch for Gateway 10.1 on Feb 8, 2022, along with a patch for Gateway 10.0 on March 4, 2022.
This patch will remove offending classes from the remaining log4j 1.x occurrences in third party libraries. Please install this patch to reduce exposure to log4j vulnerabilities.
This patch also addresses other vulnerabilities and exposures related to third-party libraries.
This patch is available: Gateway Patches Page
Please note that Monthly Platform Patches are cumulative in nature so the LATEST 10.0 or 10.1 Monthly Platform will include these fixes. You do not have to apply these older MPP's.
10.1 Patch: Layer7_API_SSG_Platform_Update_64bit_v10.1.00-CentOS_2022-02-02.L7P
10.0 Patch: Layer7_API_SSG_Platform_Update_64bit_v10.0.00-CentOS_2022-03-02.L7P
By removing these offending classes we have reduced exposure risk significantly. Even if the Gateway administrator mistakenly configures log4j within the Gateway platform to use these offending classes, the Gateway will continue to work as-is and throw exceptions while processing log statements, as shown in the following:.
log4j:ERROR Could not instantiate class [org.apache.log4j.net.JMSAppender].
java.lang.ClassNotFoundException: org.apache.log4j.net.JMSAppender not found
=================
Layer7 API Gateway 9.x/10.x (CVE-2021-44228 & CVE-2021-45046)
=================
API Gateway base functionality is not affected by the log4j. You can note the Layer7 API Gateway Security Advisory announcement. With further review, the SSO SDK which exists on appliances will require updates.
Layer7 API Gateway Appliances (Physical/virtual/software) SSO SDK:
The gateway does not use this feature but the jar file may be present within the /opt/CA folder. The SSO SDK default install location is /opt/CA. You can use this search string below to find any log4* jar files:
# find /opt/CA -name log4j*.jar
Note: These can be deleted as they are not utilized. No restart of services are required.
Layer7 API Gateway Container SSO SDK:
Our containers, post Gateway 9.4CR05 and Gateway 10 CR01, are skinny builds of SSO SDK are free from SSO log4j binaries. We suggest that anyone not running a container of these builds or greater pull the later docker images by having the tag gateway:<tag> where :<tag> would be something greater or equal to the already mentioned releases.
Specific tags can be found on Docker Hub.
Note: Custom and Tactical assertions were reviewed and no threats were determined. The Policy Manager is also not affected by this vulnerability.
Special Note: Our log4j 1.x does not use JMSAppender, and does not include the "JndiLookup.class". Customers should check the log4j configuration(s) if they have added JMSAppender and should remove them to avoid this vulnerability.
File log4j-over-slf4j-1.6.6.jar is not impacted with this CVE.
=================
Layer7 API Gateway 9.x/10.x (CVE-2019-17571)
=================
The impact of CVE-2019-17571 was investigated in the past. The gateway is not affected because the vulnerable class (SocketServer) is not being used in the GW implementation or logging configurations.
One of the few GW components containing log4j 1.2.17 are NCES assertions. If desired, you can remove this NCES assertions. On a side note, these NCES assertions will be removed in GW 10.1 CR1.
File log4j-over-slf4j-1.6.6.jar is not impacted with this CVE.
=================
Layer7 API Gateway 9.x/10.x (CVE-2021-44832)
=================
The gateway is not affected by this vulnerability as the versions of log4j impacted by this are not present on the gateway.
File log4j-over-slf4j-1.6.6.jar is not impacted with this CVE.
=================
Layer7 API Developer 4.x / 5.x (CVE-2021-44228 & CVE-2021-45046)
=================
Layer7 API Developer Portal Saas:
Similar to below (on Prem), solr containers were disabled so the risk of log4j in Saas is mitigated.
Note: Review Q&A under on Prem for details on what solr does (if required).
Layer7 API Developer Portal (on Prem):
API Developer Portal code has been reviewed and tested. At this time the only log4j issue has been observed in the solr container.
Q: What does the solr container do?
A: solr helps with giving you the TOP 5 past search results on the Portal Dashboard and API Listing pages. The current implementations we suggest disabling SOLR, which will not disable search but it will disable auto-suggesting.
Patched Mitigation (Upgrade) Steps for Layer7 API Developer Portal:
Patches for versions 4.5, 5.0.* containing updated portal.sh scripts to exclude Solr container were released in February.
We created patches for use during an upgrade. This prevents having to read use the "Manual Mitigation Steps for Layer7 API Developer Portal" below,
The Patches,
The patch versions above will address log4 vulnerability automatically.
These patch versions can be obtained at the Solutions & Patches page.
Manual Mitigation Steps for Layer7 API Developer Portal:
For Docker Swarm installation:
1) If the portal is already running, scale the Solr service to 0:
Run the following command from the OVA or the location where API Developer Portal is running:
docker service scale portal_solr=0
2) Shutdown portal
# docker stack rm portal
3) To prevent the Solr service from being started via portal.sh, make the following script change in portal.sh in function single from:
cat "$path_yml"
to:
cat "$path_yml" | sed '/solr/,/tenant-provisioner/c\ \ tenant-provisioner:'
Note: This script change removes the solr definition from the underlying Docker Compose file and prevents the Solr service from starting.
Note2: In swarm you may still note the SOLR container exists if you want complete removal (thought it will be inactive)
You can
a) Find all Solr images
docker images -a | grep solr
b) remove all images by id
docker rmi -f 1dfabcdef 1dfffffabc
Where 1dfabcdef and 1dfffffabc is the actual image id returned by step a.
c) validate no solr containers still exist.
docker images -a | grep solr
For K8s installation using Helm Charts:
Option 1: In values.yaml, set the solr replica count to 0 to remove the solr pods and push chart updates using helm upgrade command (recommended).
Example entry in values.yaml:
solr:
forceRedeploy: false
replicaCount: 0
Option 2: Manually scale down the solr pod to 0 using your Kubernetes platform user interface or kubectl commands. This option will not cause other pods redeployment However we strongly suggest to alter values.yaml as per Option 1 used for future deployment to prevent solr pod from unintentionally scaling up when using charts for upgrades/new installations.
IMPORTANT NOTE: In Portal 5.0.2 onwards, log-level settings are configurable in the druid sub-charts. The default log level is WARN. Customers are advised not to apply log level ‘DEBUG/TRACE/ALL'.
Portal 4.4, 4.5, 5.0 and 5.0 CR1 - containers that have not been specifically identified may include the log4j library. In these cases, the library is a transitive dependency. This library is not used and it has been confirmed that the containers are not vulnerable in these product versions.
For both installations, the Layer7 team will be providing updated portal.sh or values.yaml described above to remove solr in the Portal Solutions & Patches page so that these files are available when customers upgrade to those versions.
=================
Layer7 API Developer 4.x / 5.x: On Prem + SaaS (CVE-2021-4104 & CVE-2019-17571)
=================
NO IMPACT - The zookeeper and kafka containers within the analytics stack include the log4j-1.2.17.jar library. However, the containers do not use SocketServer to listen to log events and do not use JMSAppender. It has been confirmed that the containers are not impacted by the vulnerability.
=================
Layer7 API Developer 4.x / 5.x: On Prem + SaaS (CVE-2021-44832 )
=================
NO IMPACT - The product is not using JDBC Appender and has been confirmed it is not affected by this vulnerability.
=================
Layer7 Live API Creator (CVE-2021-44228 & CVE-2021-45046)
=================
The vulnerability is present in the Live API Creator. The vulnerability arises when a customer uses custom logging configuration to capture logs generated by Live API Creator.
Note: 12/29/21 - LAC patch 5.4.4 has been released which addresses the log4j vulnerability within the Live API Creator Product. More details are available within the LAC 5.4 release notes. If the 5.4.4 patch is not in place then the below mitigation steps can be followed in the interim.
Mitigation steps to custom logger:
Option 1: Change the log4j configuration file, laclogging.xml to change log output layout pattern from “%m” to “%m{nolookups}”. For example, from:
<Appender type="File" name="file" fileName="${filename}" createOnDemand="true">
<Layout type="PatternLayout">
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</Layout>
</Appender>
To
<Appender type="File" name="file" fileName="${filename}" createOnDemand="true">
<Layout type="PatternLayout">
<Pattern>%d %p %c{1.} [%t] %m{nolookups}%n</Pattern>
</Layout>
</Appender>
Option 2: Setup the environment variable “LOG4J_FORMAT_MSG_NO_LOOKUPS=true”
For more details please review External Logging for LAC.
=================
Layer7 Live API Creator (CVE-2021-4104 & CVE-2019-17571)
=================
The product is not using Apache Log4j 1.2 and JMSAppender. This been confirmed that the containers are not affected by this vulnerability.
=================
Layer7 Live API Creator (CVE-2021-44832 )
=================
The product is not using JDBC Appender. This been confirmed it is not affected by this vulnerability.
=================
Layer7 API Gateway OTK/MAG/Mag SDK
=================
These components are not affected and do not have any log4j implementation of concern.
=================
Layer7 API Performance Management (aka PAPIM)
=================
These components are not affected and do not have any log4j implementation of concern.
Additional Information
There are ongoing updates with log4j. Broadcom plans to update the libraries once log4j has stabilized their changes. Timing for updates is still TBD for now.
Feedback
