Detecting & Preventing CVE-2021-44228 (Log4Shell) with the NSX Distributed IDS/IPS
Article ID: 316657
Updated On:
Products
Issue/Introduction
This KB article covers how customers can use the NSX Distributed IDS/IPS to detect and protect against attempts at exploiting the CVE-2021-44228 vulnerability, also known as Log4shell. This Remote Code Execution (RCE) vulnerability in Apache Log4j2 allows malicious actors to load and execute arbitrary code from LDAP servers when the Log4j message lookup substitution feature has been enabled. VMware's Threat Analysis unit has overserved active exploitation attempts and is continuously monitoring adversary activities. The NSX IDS/IPS signatures we provide are continuously updated based on exploit and obfuscation techniques observed. To read more about the analysis by the VMware Threat Analysis Unit, check out this blog https://blogs.vmware.com/security/2021/12/log-in-the-shell-an-analysis-of-log4shell-exploitation.html.
Note: The IDS/IPS feature is available through a subscription license for both NSX-T on prem (Intrusion Detection in NSX-T 3.0 and Intrusion Prevention in NSX-T 3.1) and VMware Cloud on AWS (VMC) using SDDC 1.15+ and above.
While this article specifically covers the NSX IDS/IPS, other controls and solutions can/should also be leveraged to limit the attack surface and detect/prevent post-exploit activity such as lateral movement or deployment of ransomware or crypto miners. In addition, software vendors are providing workarounds and patches to remediate the issue, as well broad recommendations for any software leveraging vulnerable versions of open-source Apache Log4j library.
The NSX Distributed and Gateway Firewall should be leveraged to restrict outbound communication associated with the exploit. Specifically, LDAP and RMI communication should only be allowed to trusted servers based on an allow-list approach. In addition, the distributed firewall should also be used to prevent post-exploit activity and lateral movement beyond the initial target. NSX Network Detection & Response (NDR) and Network Traffic Analysis (NTA) can further identify traffic anomalies associated with unusual post-exploit activity and correlate this activity with the initial exploit. To learn more about how other VMware products including the NSX Advanced Load Balancer (AVI) and Carbon Black Endpoint & NDR can help, please check out https://blogs.vmware.com/security/2021/12/investigating-cve-2021-44228-log4shell-vulnerability.html
Environment
Resolution
As the distributed IDS/IPS is applied to the network interface (vNIC) of a workload, we can provide intrusion detection and prevention for every workload regardless of the physical location (which hypervisor), the origination (external/internal attacker) or the tactic (i.e., initial access, discovery, lateral movement, exfiltration) being leveraged.
Several NSX IDS/IPS Signatures have been released to detect and prevent network activity associated with attempts of exploiting this vulnerability. This activity could be indicative of scanning (which accounts for the majority of observed associated events at the time of writing) or real exploitation attempts. The signatures we provide detect the "JNDI ${}" string which triggers the exploit, as well as related inbound and outbound network activity including LDAP and RMI and obfuscation techniques known to be used with this exploit. At the time of writing of this KB article, 28 IDS/IPS signatures associated with CVE-2021-44228 are available NSX customers. This includes signatures released by Emerging Threat Labs (ET) as well as SpiderLabs Research (SLR).
Below are the available signatures at the time of writing. Additional signatures may be provided through signature set updates delivered from the NSX Threat Intelligence cloud.
SpiderLabs Research:
4103226 SLR Alert - Apache Log4j Log4Shell RCE HTTP LDAP Attempt (CVE-2021-44228) rev:1
4103227 SLR Alert - Apache Log4j Log4Shell RCE TCP LDAP Attempt (CVE-2021-44228) rev:1
4103228 SLR Alert - Apache Log4j Log4Shell RCE HTTP LDAPS Attempt (CVE-2021-44228) rev:1
4103229 SLR Alert - Apache Log4j Log4Shell RCE TCP LDAPS Attempt (CVE-2021-44228) rev:1
4103230 SLR Alert - Apache Log4j Log4Shell RCE HTTP RMI Attempt (CVE-2021-44228) rev:1
4103231 SLR Alert - Apache Log4j Log4Shell RCE TCP RMI Attempt (CVE-2021-44228) rev:1
4103232 SLR Alert - Apache Log4j Log4Shell RCE HTTP DNS Attempt (CVE-2021-44228) rev:1
4103233 SLR Alert - Apache Log4j Log4Shell RCE TCP DNS Attempt (CVE-2021-44228) rev:1
Emerging Threat:
2034647 ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228) rev:1
2034648 ET EXPLOIT Apache log4j RCE Attempt (http rmi) (CVE-2021-44228) rev:1
2034649 ET EXPLOIT Apache log4j RCE Attempt (tcp ldap) (CVE-2021-44228) rev:1
2034650 ET EXPLOIT Apache log4j RCE Attempt (tcp rmi) (CVE-2021-44228) rev:1
2034651 ET EXPLOIT Apache log4j RCE Attempt (udp ldap) (CVE-2021-44228) rev:2
2034652 ET EXPLOIT Apache log4j RCE Attempt (udp rmi) (CVE-2021-44228) rev:2
2034653 ET EXPLOIT Apache log4j RCE Attempt (udp dns) (CVE-2021-44228) rev:2
2034654 ET EXPLOIT Apache log4j RCE Attempt (tcp dns) (CVE-2021-44228) rev:2
2034655 ET EXPLOIT Apache log4j RCE Attempt (http dns) (CVE-2021-44228) rev:2
2034656 ET EXPLOIT Apache log4j RCE Attempt (udp ldaps) (CVE-2021-44228) rev:2
2034657 ET EXPLOIT Apache log4j RCE Attempt (tcp ldaps) (CVE-2021-44228) rev:2
2034658 ET EXPLOIT Apache log4j RCE Attempt (http ldaps) (CVE-2021-44228) rev:2
2034659 ET EXPLOIT Apache log4j RCE Attempt - lower/upper TCP Bypass (CVE-2021-44228) rev:1
2034660 ET EXPLOIT Apache log4j RCE Attempt - lower/upper UDP Bypass (CVE-2021-44228) rev:2
2034667 ET EXPLOIT Apache log4j RCE Attempt (udp iiop) (CVE-2021-44228) rev:2
2034668 ET EXPLOIT Apache log4j RCE Attempt (tcp iiop) (CVE-2021-44228) rev:2
2034671 ET EXPLOIT Apache log4j RCE Attempt - 2021/12/12 Obfuscation Observed M1 (CVE-2021-44228) rev:1
2034672 ET EXPLOIT Apache log4j RCE Attempt - 2021/12/12 Obfuscation Observed M1 (CVE-2021-44228) rev:1
2034673 ET EXPLOIT Possible Apache log4j RCE Attempt - 2021/12/12 Obfuscation Observed M2 (CVE-2021-44228) rev:1
2034674 ET EXPLOIT Possible Apache log4j RCE Attempt - 2021/12/12 Obfuscation Observed M2 (CVE-2021-44228) rev:1
Downloading the latest signature set:
NSX Manager can be configured to automatically download the latest IDPS signature set from the NSX Threat Intelligence Cloud and propagate it to the individual transport nodes (hypervisors) on which the distributed IDPS has been enabled on. Alternatively, for customers how have deployed NSX in an air-gapped environment, customers can download the signatures manually from our threat intelligence cloud and upload the bundle to NSX manager. The offline download mechanism is described here in the NSX-T Admin guide.
Image: select the latest signature set as the active set (not required when the "auto update new versions" feature has been enabled.
Viewing the relevant signatures
Once the latest signature set has been downloaded and selected as the active set, you can confirm the signatures listed above are now available via Global Signature Management.
Click "View and manage global signature set>>" and then apply a filter based on CVE 2021-44228. You should see the below list of signatures.
Image: Apply a CVE-based filter in Global Signature Management to confirm signatures for CVE 2021-44228 are available.
Note: The number of signatures available to cover this vulnerability depends on the signature set version. You may see a higher number than the number shown here at time of writing this article.
Creating an IDPS profile that includes these signatures:
The next step is to create an IDPS profile that includes these signatures. Since all these signatures have a severity of "HIGH", and attack target of "Server", one option is to create a profile that includes all signatures with severity of "HIGH" and "CRITICAL" and Attack Target of "Server".
Image: Create an IDPS profile to include the signatures related to this vulnerability.
Click “Manage signatures for this profile >>" and filter based on CVE 2021-44228 to confirm these signatures are included in the profile. Note that these signatures come with a default action set to "Alert". For the NSX Distributed IDS/IPS to drop the offending traffic in addition to alerting, you will need to change the action to either "Drop" or "Reject". Note that the distributed IDS/IPS will only take a preventative action when it's deployed in "detect & prevent" mode.
Image: Verifying signatures are included in the IDPS profile and changing the action to "Drop"
Note: If you already have an IDPS profile created, based on the inclusion criteria these signatures may already be included. The severity for all these signatures is set to "HIGH". You can click "Manage signatures for this profile >>" within a profile to confirm the signatures are included.
Creating workloads groups and an IDPS policy:
The next and final configuration step is to create an IDPS policy and rule(s). These define where the policy you just configured will be applied to.
Optionally, if you know (i.e., as a result of vulnerability assessment) what workloads in your environment are running a service that has implemented a vulnerable version of Apache log4j logging service, you can apply the IDPS profile specifically to those workloads. To do so, apply a Tag to each vulnerable workload, and create a group that includes the tagged workloads. Alternatively, if you have not established the scope of vulnerable services, you can apply the IDPS profile more broadly to the workloads in your environment.
Image: Creating a new security tag
Image: Tagging potentially vulnerable virtual machines
Image: Creating a Group based on the tag
Once you have created groups, you can define create the IDPS policy and rule(s). Set the "Applied To" for these rules to the security group you just created. In order take preventative action in addition to alerting when an exploit is attempted, set the mode to "Detect & Prevent". In addition, as mentioned earlier, the signature action also needs to be changed to "drop" or "reject" for the NSX distributed IDS/IPS to take a preventative action. To get alerted when a signature is triggered, set the action to "Detect only".
.
Image: Creating an IDPS policy and rule "applied-to" set to the group you created earlier, applying the IDPS profile in "Detect & Prevent" mode.
Note: If you already have an IDPS policy and rules created that already cover the potentially vulnerable servers, you don't need an additional rule. In this case, do ensure that the appropriate signatures have been enabled in the profile applied in the rule.
Monitoring Log4Shell activity:
When an IDPS signature is triggered on a distributed IDS/IPS instance running on a hypervisor transport node, the event is sent up to the NSX manager where it can be consumed from the NSX UI. Alternatively, you can also enable syslog event export to send IDPS events directly from the hypervisor transport node to your SIEM/syslog collector.
To monitor 2021-44228 related activity using the NSX Manager UI, navigate to Distributed IDS/IPS → Events. Here you can create a filter set to "2021-44228" to display all signature events related to this vulnerability. You can also change the timeline to look at events across all distributed IDS/IPS instances for the last 24 hours - last 14 days. Additional details such as the Source, Destination IP address, affected VMs, whether the attempt was prevented and details about the vulnerability are available from each individual event.
Image: Monitoring IDS/IPS events related to CVE 2021-44228 from the NSX UI.
Feedback
