Forged in Fire: A Survey of MobileIron Log4Shell Exploitation
On December 10, 2021, the Apache Software Foundation disclosed CVE-2021-44228, aka “Log4Shell”, a critical vulnerability in Apache’s Log4j version 2.14.1 and earlier that affects a large number of products that utilize this logging library.
Through our Consulting and Managed Defense clients, Mandiant observed four unique applications targeted and exploited using CVE-2021-44228. One product that caught our attention in the immediate aftermath of this CVE’s release was MobileIron Core; an on-premises mobile device management solution owned by Ivanti, who immediately responded to the vulnerability and proactively informed customers using the impacted products, recommending that they follow test mitigations within 24 hours. Proof-of-concept (PoC) code quickly became available to exploit the vulnerability in unpatched MobileIron systems and can be easily modified to target any organization using the product. As a result, we observed a few notable adversaries jump at the opportunity.
In this blog post, we will discuss the exploitation of MobileIron servers by four unique adversaries, providing insight into each group’s operations and capabilities. While the initial exploitation of a vulnerable MobileIron server appeared the same across intrusions, our methodical clustering and delineation of post-exploitation actions unveiled some familiar adversaries and unmasked some new players that made quick use of this newly published exploit.
CVE-2021-44228 Big Picture
First let’s take a step back and review Log4j targeting since CVE-2021-44228's release. Beginning the day of that initial disclosure, Mandiant observed mass scanning and exploitation attempts across a large variety of our customers. Financially motivated actors immediately jumped at the opportunity to deploy cryptominers such as XMRIG. In mass exploitation scenarios, like Log4Shell, we have found that cryptominers often drop first. While not immediate, Mandiant later observed ransomware actors exploit CVE-2021-44228 to gain their initial foothold into target environments. Groups that are widely suspected of being related to foreign intelligence entities, such as APT41, wasted no time in exploiting MobileIron servers as they were observed within a day of the announcement of the vulnerability.
Since its initial disclosure, we have observed 22 distinct clusters of activity involved in the exploitation of CVE-2021-44228. These adversaries used a range of tools and techniques in support of varying mission objectives during the post-exploitation phase of their intrusions. Table 1 shows 12 of those clusters sorted by their assessed goals who have conducted Log4Shell-based campaigns against multiple organizations.
Financial, Gov’t, Higher Ed, Telecom
Business Services, Financial, Food Services, Gov't, Healthcare, Higher Ed, Media, Retail, Transportation
Business Services, Financial, Food Services, Gov't, Higher Ed, Retail, Technology, Telecom
Through all the noise of Log4j scanning, cryptominers, security testing, and intrusion activity that occurred in December 2021, the following four threat groups exploited MobileIron to conduct the most significant intrusions observed by Mandiant.
|UNC961||Unknown||Financial Gain||UNC961 Profile|
Mandiant uses the label “UNC” group—or “uncategorized” group—to refer to a cluster of intrusion activity that includes observable artifacts such as adversary infrastructure, tools, and tradecraft that we are not yet ready to give a formal classification to, like APT or FIN (learn more about how Mandiant tracks uncategorized threat actors).
Further details surrounding each of the aforementioned groups are provided in Table 3:
Threat Group Details
APT41 is a Chinese state-sponsored espionage group that also conducts financially motivated activity for personal gain. The group has been active since at least 2012 and has conducted espionage operations against a wide number of industries in the public and private sectors. The group has executed multiple supply chain compromises that allowed them to gain access to software companies and inject malicious code into legitimate files before distributing updates. APT41 also carried out operations against the video game industry for financially motivated intrusions as well as to steal source code and digital certificates.
Most recently, APT41 has spent considerable time and resources to conduct a long-term campaign against U.S. state government entities using web exploits to target public-facing servers. APT41’s participation in Log4Shell exploitation is a logical continuation of their past year of activity. For more details on APT41 and their persistent campaign targeting U.S. state government computer networks, see our recent blog post: “Does This Look Infected?”: A Summary of APT41 Targeting U.S. State Governments
Mandiant has tracked UNC961 since January 2018, with confirmed activity dating back to November 2016. UNC961 is a financially motivated group that has historically targeted organizations in the retail and business service sectors in North America. Starting in mid-2020, UNC961 expanded its targets to health care, energy, financial transactions, and high-tech organizations in North America, Northern Europe, and Western Asia. In all known activity, UNC961 has utilized web exploits to initiate their intrusions, having targeted Atlassian Confluence, ForgeRock AM, and Oracle Web Logic servers.
Mandiant clustered UNC3500 in the immediate aftermath of Log4Shell’s public release. Mandiant suspects UNC3500 has a China nexus and has been observed targeting entities in the education and telecommunications sectors. We confirmed overlaps between UNC3500’s Log4Shell activity and a set of suspected Chinese espionage activity that Mandiant clustered in May 2021. These overlaps included the use of SoftEther VPN to maintain access to a system, specific SoftEther configuration settings, related infrastructure, and targeting overlaps.
Mandiant clustered UNC3535 in the aftermath of Log4Shell’s public release. While we have not gathered enough information on UNC3535 to assess their primary motivation, UNC3535 has exfiltrated sensitive data from organizations in the transportation sector.
In this section we walk through the MobileIron-based intrusions conducted by each of the aforementioned adversaries.
In one of the most notable intrusions, APT41 made quick use of this public exploit to target organizations, which included multiple U.S. state government entities. APT41 used CVE-2021-44228 to target at least four organizations: a telecom company based in the APJ region that is a frequent target of Chinese espionage operations, one US-based financial organization, and two state level government agencies in the U.S.
APT41 leveraged CVE-2021-44228 against vulnerable MobileIron servers to launch reverse shell payloads back to their controlled server. Using this access, APT41 executed commands on the system under the context of the user account tomcat. One example reverse shell payload is presented in Figure 1.
The base64-encoded string decodes to the following reverse shell payload (Figure 2).
Mandiant also observed payloads consistent with verifying network connectivity to a threat actor-controlled host (Figure 3).
Approximately one hour later, Mandiant identified further exploitation attempts using sub-domains of eu[.]org to test outbound network connectivity (Figure 4).
After confirming outbound connectivity, APT41 used CVE-2021-44228 to execute a new reverse shell payload (Figure 5).
Once APT41 established the reverse shell, they leveraged wget to download and stage their payload on the MobileIron server (Figure 6).
Mandiant determined the kernel file was an ELF variant of KEYPLUG, tracked now as KEYPLUG.LINUX. The threat actor configured the sample to connect to the domain microsoftfile[.]com for command and control. At the time of the activity, the domain resolved to the IP address 103.224.80[.]44. APT41 also modified the permissions of the binary to give full read/write/execute permissions to all users (Figure 7).
Finally, APT41 took a few steps to attempt to hide their activity. They renamed the KEYPLUG.LINUX binary to .kernel to hide the file with the hidden file attribute, then leveraged the nohup utility to ensure the process remains running and ignores hangups. APT41 also leveraged the & operator to execute the binary as a background process (Figure 8).
Mandiant Managed Defense’s swift detection and containment of this intrusion once again proved its worth against even the most advanced and prolific espionage actors. However, as noted in our recently released blog post on APT41’s latest campaigns, the group has remained undeterred by the U.S. Department of Justice (DOJ) indictment in September 2020 and continues to operate at an ever-increasing tempo with a focus on vulnerable U.S. state and local government networks. Mandiant expects the group to continue to evolve and develop their own attack vectors while integrating existing capabilities into their toolkit.
UNC961 is a suspected financially motivated threat group that Mandiant has tracked since 2018. They seized the opportunity to exploit CVE-2021-44228 by targeting organizations with publicly accessible MobileIron servers. UNC961 is notable for their seemingly exclusive use of exploits against web applications for initial access.
At one target, Mandiant observed UNC961 use their CVE-2021-44228 payload to establish a reverse shell. This payload was also crafted to unset the HISTFILE environment variable to thwart forensic analysis by preventing the command line history from being written to disk.
Figure 10 shows the deobfuscated exploit string.
The decoded command, shown in Figure 11, removed command-line logging and executed a reverse shell payload.
UNC961 commonly employs this anti-forensic technique in their Linux-based intrusions, either as a component of an exploit payload such as here with Log4Shell, or during the hands-on-keyboard post-exploitation phase. Following their initial foothold, they performed host-based reconnaissance, and the following day deployed their HOLEPUNCH tunneler. HOLEPUNCH is a Windows and UNIX source-compatible utility that uses SOCKS5 style commands to multiplex connections back to its command and control (C2) server.
It is worth noting that, in addition to targeting MobileIron servers, UNC961 also exploited a VMWare Horizon Server belonging to a separate victim during their Log4Shell campaign. Following the Horizon Server’s exploitation, UNC961 deployed two previously unobserved backdoors: HOLEDOOR and DARKDOOR. HOLEDOOR is written in C, whereas DARKDOOR is written in Go. UNC961 used these backdoors to conduct reconnaissance in the victim environment and steal credentials for users who had previously logged in to the VMware Horizon server. Additionally, UNC961 exported the contents of the SAM, SECURITY, and SYSTEM registry hives using the built-in Windows tool reg.exe. UNC961 then exfiltrated the exported registry data to their infrastructure using the PSCP utility.
One day after the initial Apache announcement, UNC3500 jumped at the opportunity to target an educational institution in North America. Having successfully weaponized the Log4Shell exploit, UNC3500 launched a reverse shell on a MobileIron server. After performing some initial host reconnaissance, the adversary achieved persistence through a relatively unique method. Using the built-in curl command, UNC3500 downloaded three files, modified permissions of one, and launched another as a new process (Figure 12).
The file's https (MD5: 00352d167c44272dba415c36867a8125) and hamcore.se2 (MD5: 9fb1191ba0064d317a883677ce568023) are components of SoftEther’s VPN server bridge, PacketiX. The PacketiX VPN Bridge creates a layer 2 connection between a physical network adapter on a local system and a remote SoftEther VPN server. It requires an accompanying library file hamcore.se2 and a configuration file vpn_bridge.config. By deploying this package, UNC3500 established persistence on the compromised MobileIron server.
UNC3500 appeared to have some difficulty initially establishing their VPN. They downloaded their vpn_bridge.config file not once but four times, each with minor changes. Table 4 shows the three uniquely configured C2 servers from these files.
UNC3500 had to troubleshoot their connection issues by examining VPN log files and running ping, route, and curl commands (Figure 13).
UNC3500 used the final configuration file to establish a VPN connection with a SoftEther server hosted on 45.76.98[.]184 over port 443 using a unique certificate and key combination. UNC3500 proceeded to hide their tracks by clearing the bash history using the command history –c.
Analysis of these config files identified overlaps with a prior intrusion where this group targeted an organization in the telecommunications vertical. During these prior on-host intrusions, Mandiant observed UNC3500 engage in reconnaissance activities primarily using Windows built-in commands or PSEXEC, perform credential harvesting using MODKATZ, and deploy BEHINDER and CHOPPER web shells.
Finally, Mandiant clustered another unique set of MobileIron exploitation activity as UNC3535. Mandiant observed UNC3535 use CVE-2021-44228 to deploy a reverse shell and use their access to harvest and exfiltrate sensitive MobileIron data.
As typical with other Log4Shell exploits, the adversary used a base64-encoded string (Figure 14).
The base64-encoded string in Figure 14 decodes to Figure 15.
This command attempted to establish a bash reverse shell to the specified external server. Over the following half-hour, six connections to the IP address 187.109.15[.]2:443 were initiated.
Approximately 21 minutes after launching the first reverse shell, the adversary issued the following HTTP GET request to the MobileIron server.
Initial analysis of the captured network traffic determined this file to contain approximately 154MB of data.
Ten minutes following this GET request, the threat actor modified the directory /mi/tomcat/webapps/mifs/images. At the time of analysis, no files within this directory were dated near that last modified (mtime) timestamp. This evidence is consistent with the deletion of a file, likely wtower_in.png, from the directory. Mandiant performed forensic analysis of the server disk image and found that the file was overwritten and was therefore unable to be recovered. However, Mandiant found evidence that the attacker dumped the MobileIron “MIFS” database five minutes prior to the HTTP GET request. The MIFS database contains sensitive data, including device information, password history, and other data necessary for device management.
Strings indicative of a MySQL database dump were also found in the free space of the disk image:
The speed at which this intrusion operation occurred indicates the threat actor’s existing knowledge of the MobileIron ecosystem. Swift actions to identify, archive, and stage the targeted sensitive data resulted in successful data theft.
The release of Log4Shell in mid-December 2021 capped off the worst year on record for the exploitation of zero-day vulnerabilities. We can only expect this trend to continue, with PoCs becoming available within mere hours of a vulnerability's disclosure, like in Log4Shell’s case, furthering the need for diligent and proactive asset and patch management.
Adversaries of every category will continue to exploit CVE-2021-44228 in unpatched systems for the foreseeable future due to Apache Log4j integration in unknown numbers of applications. Ivanti and Mandiant highly recommend organizations patch this vulnerability in their environment to protect themselves against attacks that aim to exploit this issue. Leveraging Mandiant's Attack Surface Management platform, organizations can better identify vulnerable Log4j instances in their environment.
Mandiant Security Validation Actions
Organizations can validate their security controls using the following actions with Mandiant Security Validation.
Application Vulnerability - CVE-2021-44228, HTTP GET, DNS Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via Referer
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via URI Path
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent, Base64 Variant #1
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent, Environmental Variable Variant #1
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent, Environmental Variable Variant #2
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent, Obfuscated Variant #1
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAP Callback via User-Agent, Obfuscated Variant #2
Application Vulnerability - CVE-2021-44228, HTTP GET, LDAPS Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTP GET, RMI Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTP POST, DNS Callback via Username
Application Vulnerability - CVE-2021-44228, HTTP POST, LDAP Callback via Username
Application Vulnerability - CVE-2021-44228, HTTP POST, LDAPS Callback via Username
Application Vulnerability - CVE-2021-44228, HTTP POST, RMI Callback via Username
Application Vulnerability - CVE-2021-44228, HTTPS GET, DNS Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTPS GET, LDAP Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTPS GET, LDAPS Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTPS GET, RMI Callback via User-Agent
Application Vulnerability - CVE-2021-44228, HTTPS POST, DNS Callback via Username
Application Vulnerability - CVE-2021-44228, HTTPS POST, LDAP Callback via Username
Application Vulnerability - CVE-2021-44228, HTTPS POST, LDAPS Callback via Username
Application Vulnerability - CVE-2021-44228, HTTPS POST, RMI Callback via Username
Evaluation CVE-2021-44228, log4j, Network Actions
Code Family Definitions
KEYPLUG.LINUX is a Linux backdoor written in C++ that can communicate via TCP over UDP or via TCP. Its core functionality involves expanding its capabilities by downloading plugins from a hardcoded C2 server. Plugins are mapped directly into memory and executed.
HOLEPUNCH is a Windows & UNIX source-compatible tunneler utility that uses SOCKS5 style commands wrapped in a custom outer structure to multiplex connections back to its C2 server. The program may apply an XOR hardcoded value with data chunks transferred. The SOCKS5 commands received by HOLEPUNCH cause it to initiate new connections to the host or IP address specified by the operator, and then begin transferring data back & forth with the remote system.
HOLEDOOR is a 64-bit Windows backdoor that is written in C and communicates via TCP. It is capable of spawning reverse shells that execute commands via cmd.exe. It supports some basic commands to spawn a new shell, disconnect any current running shells, or terminate an entire application.
DARKDOOR is a backdoor written in Go that is highly modular in design. It supports communication over TLS and HTTP. It has capabilities to execute arbitrary code and list running processes.
All the Mandiant analysts and consultants from our Managed Defense and Incident Response teams around the world, the analysts and researchers from Advanced Practices and Mandiant Intelligence, FLARE Reverse Engineers, and everyone involved in the monumental effort undertaken during the fallout of CVE-2021-44228's disclosure.
We would also like to acknowledge Ivanti for coordinating with us on this blog’s release and for their quick response to this unprecedented vulnerability.