Introduction

FireEye iSIGHT Intelligence compiled extensive data from dozens of ICS security health assessment engagements (ICS Healthcheck) performed by Mandiant, FireEye's consulting team, to identify the most pervasive and highest priority security risks in industrial facilities. The information was acquired from hands-on assessments carried out over the last few years across a broad range of industries, including manufacturing, mining, automotive, energy, chemical, natural gas, and utilities. In this post, we provide details of these risks, and indicate best practices and recommendations to mitigate the identified risks.

Mandiant ICS Healthchecks

Mandiant ICS Healthchecks and penetration testing engagements include on-site assessments of customers' IT and ICS systems. The ICS Healthcheck consists of workshops and technical reviews. It captures the results in a final report that ranks discovered findings and vulnerabilities by risk using Mandiant’s Risk Rating method. During an onsite workshop with site technical experts, Mandiant develops a technical understanding of the subject control system(s), builds a network diagram of the control system, analyzes for potential vulnerabilities and threats, and assists with prioritizing recommended countermeasures to defend the environment.

Mandiant also collects and reviews packet captures of network traffic from the ICS environment to validate the network diagram constructed in the workshop and to identify any unexpected or undesirable deviations from the intended design. This traffic is also analyzed for evidence of compromise or misconfiguration of the ICS network/system. Mandiant inspects the deployed security technology for vulnerabilities and other architectural risks, such as inappropriately configured firewalls, dual-homed control system devices, and unnecessary connectivity to the business network or the Internet.

NOTE: Findings are discussed at a generalized level to preserve the anonymity of our customers. This post presents a high-level overview and is meant to be an informative first stop for customers interested in common cyber security issues. For more information or to request Mandiant services, please visit our website.

Methodology: Mandiant Risk Rating System

This blog post leverages information from Mandiant ICS Healthchecks, which evaluate cyber security risk in organizations from multiple industries. The rating of critical and high security risk is based on the Mandiant Risk Rating System, which is determined by identifying the exploitability and the impact of a given issue, and cross-referencing the results (Figure 1).

One Third of Security Risks in ICS Environments Ranked High or Critical

We reviewed findings from all of our risk assessments and then categorized and ranked the reported risks as critical or high, medium, low, or informational (Figure 2). At least 33 percent of the security issues we found in ICS organizations were rated of high or critical risk. This means they were most likely to allow adversaries to readily gain control of target systems and potentially compromise other systems or networks, cause disruption of services, disclose unauthorized information, or result in other significant negative consequences. We suggest immediate remediation for critical risks, and quick action to remediate high security risks.

Most Common High and Critical Security Risks in ICS Environments

FireEye iSIGHT Intelligence organized the critical and high security risks identified during Mandiant ICS Healthchecks into nine unique categories (Table 1). The three most common were:

• Vulnerabilities, Patches, and Updates (32 percent)
• Identity and Access Management (25 percent)
• Architecture and Network Segmentation (11 percent)

In most of these cases, basic security best practices would be enough to stop (or at least make it more difficult for) threat actors to target an organization's systems. The implications are vast because specialized malware or actors targeting infrastructure would likely look for these flaws first to exploit throughout the targeted attack lifecycle.

Top Three High and Critical Risks and Recommended Mitigations

Vulnerabilities, Patches, and Updates

Vulnerability, patch, and update management procedures enable organizations to secure off-the-shelf software, hardware, and firmware from known security threats. Known vulnerabilities in ICS environments can be leveraged by threat actors to access the network and move laterally to execute targeted attacks. The following common risks were observed during our engagements:

• Infrequent procedures for patching and updating control systems:
  • We encountered organizations with no formal vulnerability and patch management programs.
• Out-of-date firmware, hardware, and operating systems (OS), including:
  • Network devices and systems such as switches, firewalls, and routers.
  • Hardware equipment, including desktop computers, cameras, and programmable logic controllers (PLCs).
  • Unsupported legacy operating systems such as Windows Server 2003, XP, 2000, and NT 4.
• Unaddressed known vulnerabilities in software applications and equipment where patches are available:
  • We observed outdated firewalls with up to 53 unaddressed vulnerabilities and switches with more than 200 vulnerabilities.
  • System management software that can be exploited using known open source tools.
• Lack of test environments to analyze patches and updates before implementation.

Mitigations

• Develop a comprehensive ICS Vulnerability Management Strategy and include procedures to implement patches and updates on key assets. More information is provided by the National Institute for Standards and Technology's (NIST) Guide for ICS Security NIST SP800-82.
• When patches and updates are no longer provided for key infrastructure, choose one of the two following options:
  • Implement a security perimeter around affected assets, protected by, at minimum, a firewall (industrial protocol inspection/blocking if appropriate) for access control and traffic filtering.
  • Decommission legacy devices that might be exploited to gain access to the network, such as switches.
• Set up development systems or labs that are representative of the running IT and ICS devices. These systems can often be built from existing spares along with the purchase or loan of additional licenses for human-machine interfaces (HMIs) and configuration software from the system vendor. A development system is an excellent platform to test changes and patches, and on which to perform vulnerability scans without risk to active systems.

Identity and Access Management

The second most common category of security issues identified was related to the flaws in or absence of best practices for handling passwords and credentials. Common weaknesses identified by Mandiant include:

• Lack of multi-factor authentication for remote access and critical accounts:
  • Users were able to remotely access ICS environments from the corporate network without requiring multi-factor authentication.
• Lack of a comprehensive and enforced password policy:
  • Weak passwords with insufficient length or complexity used for privileged accounts, ICS user accounts, and service accounts.
  • Passwords were not changed frequently.
  • Passwords were reused for multiple accounts.
• Prominently displayed passwords:
  • Passwords were written on the chassis of devices.
• Hard-coded and default credentials in applications and equipment:
  • Mandiant discovered Remote Terminal Units (RTUs) containing default credentials, which are commonly available on the Internet and in the device manuals.
  • A modem contained a backdoor account incorporated by the manufacturer.
• Commonly used “administrator” accounts.
• Use of shared credentials.

Mitigations

• Implement two-factor authentication for all possible users, especially administrative accounts.
• Avoid keeping written copies of passwords and, if necessary, secure them out of sight with limited access for only authorized users.
• Enforce password policies that require strong passwords that are regularly modified and cannot be reused. More information is available from SANS.
• Avoid common, easily guessed user account names such as "operator," "administrator," or "admin." Instead, use uniquely named user accounts for all access.
• Require administrative users to log in with uniquely named user accounts with strong passwords, tied back to an individual person.
• Avoid shared accounts when feasible. However, if present, they should be hardened using strong passwords that are stored in an encrypted password manager.

Network Segregation and Segmentation

Of the top three risks identified in this post, weaknesses in network segregation and segmentation are the most important. Lack of segregation from the corporate IT network and within the ICS network allows threat actors opportunities to launch remote attacks against key infrastructure by moving laterally from IT services to ICS environments. Furthermore, it increases the risk of commodity malware spreading to ICS networks where the malware could interact with operational assets. The main risks identified by Mandiant included:

• Plant systems accessible from the corporate network, either directly or through bridge devices (connected to both networks), such as unused servers, HMIs, historians, or loosely configured shared firewalls. We also found:
  • Unfiltered access to plant servers from corporate networks through, for example, a historian communicating with the distributed control system (DCS).
  • Missing segmentation between ICS and corporate networks.
  • Vulnerabilities in bridge devices (e.g., outdated appliances running vulnerable OS) that can enable lateral movement between networks.
  • Business functions (e.g., data backups and anti-virus updates) running on shared control system networks.
• Dual-homed systems, both servers and desktop computers.
• Industrial networks connected directly to the internet.

Mitigations

• Segment all access to ICS with a network Demilitarized Zone (DMZ), as recommended by both NIST SP 800-82 and IEC (Figure 3):
  • Restrict the number of ports, services, and protocols used to establish communications between the ICS and corporate networks to the least possible to reduce the attack surface.
  • Terminate incoming access for both regular and administrative users first in the DMZ, and then establish another session with connectivity into the ICS network.
  • Place servers (or mirrored servers) that provide ICS data to the corporate network in the DMZ.
  • Use firewalls to filter all network traffic entering or leaving the ICS.
  • Firewall rules should filter both incoming traffic from the corporate network and outgoing traffic from the ICS, and they should only allow the minimum required amount of traffic to pass.
• Isolate the control networks from the internet. A separate network should be used for internet access through a DMZ, and at no time should a bridged connection be allowed between the two networks.
• Ensure that independent, regularly patched firewalls are used to separate the corporate network from the DMZ and ICS network, and review firewall rulesets on a regular basis.
• Identify and redirect any non-control system traffic traversing the industrial network.
• Eliminate all dual-homed servers and hosts.

Additional Highlights

Additional common risks were identified from other categories, but with less frequency.

Network Management and Monitoring

• We identified the lack of Network Security Monitoring, Intrusion Detection, and Intrusion Prevention in organizations, including missing endpoint malware protection, leaving unused ports active, and having limited visibility into ICS networks. We recommend the following best practices:
  • A comprehensive network security monitoring strategy should be defined and implemented at the ICS level as part of an overarching ICS security program. Special attention should be placed on monitoring network segments where external connectivity occurs:
• Implement or increase centralized system and network logging to provide visibility across the entire enterprise (IT and ICS). Monitor logs for anomalous behavior. Consider implementing additional host or network-based security controls that generate alerts or reject traffic based on anomalous or suspicious behavior.
• Install a centrally managed anti-malware solution on all ICS and ICS DMZ hosts. Ensure that signature and application updates are deployed in a timely manner.
• Explore alternatives for the deployment of an advanced endpoint protection solution that provides detection/prevention for malware and malicious activities that do not rely on signature-based detection methods.
• Develop procedures to identify and shut down network ports when not in use.

Misconfigurations in Firewall Rules

We identified weak firewall rules including "ANY-ANY" configurations, conflicting or overlapping rules, overly permissive conditions allowing access to administrative services, and lack of console connection timeouts. We recommend the following best practices for secure firewall configuration:

• Filtering rules should only allow access from/to specific source/destination IP addresses and ports.
• Filter rules should specify a specific network protocol.
• ICMP filter rules should specify a specific message type.
• Filter rules should drop network packets instead of rejecting them.
• Filter rules should perform a specific action and not rely on a default action.
• Administrative session timeout parameters should be set to terminate those sessions after a predetermined amount of time.

Cyber Security Governance Best Practices

We identified some organizations with limited or absent formal and comprehensive ICS security programs. We highly suggest organizations implement ICS security programs to prioritize the following recommendations:

• Establish a formal ICS security program with a clearly defined owner, accountability, and governance structure. It should include:
  • Business expectations, policies, and technical standards for ICS security.
  • Guidance on proactive security controls (e.g., implementation of patches and updates, change management, or secure configurations).
  • Incident Response, Disaster Recovery, and Business Continuity plans.
  • ICS security awareness training plans.
• Develop a Vulnerability Management Strategy following NIST SP800-82, including asset identification and inventory, risk assessment and analysis methodology (with prioritization of critical assets), remediation testing, and deployment guidelines.

Conclusion

This blog post presents a broad picture of the current risks facing industrial organizations as observed during Mandiant ICS Healthchecks. While the trends observed in this research align with risk areas commonly discussed in security conference talks and media reports, this blog draws from dozens of on-site assessments that hold real-life validity.

Our findings indicate that at least one third of the critical and high security risks in ICS are related to vulnerabilities, patches, and updates. Known vulnerabilities continue to represent significant challenges for ICS owners that must oversee the daily operation of thousands of assets in complex industrial environments. It is also relevant to highlight that some of the most common risks we identified could be mitigated with security best practices, such as enforcing a comprehensive password management policy or establishing detailed firewall rules. If you are interested in more information or to request Mandiant services, please visit our website.