No one wanted to talk about cybersecurity for years. Keeping your head down—preferably underground—was the go-to cybersecurity strategy. Unfortunately, for many process operations and their clients, "security by obscurity" remain their default positions, even though minimally networked systems with just one or a few undocumented Ethernet port or wireless connections are continually pelted just like larger systems by a constant barrage of unauthorized probes, malware intrusions and cyber attacks. Obscurity—and most air gaps, for that matter—are illusions because they don't really exist or make process applications secure.
"Too many people still bury their heads in the sand and ignore cybersecurity problems," says Sam Hoff, CEO of Patti Engineering, a CSIA-certified system integrator in Auburn Hills, Mich. "Unless their equipment goes down, they think they have no issues. However, they could have a cybersecurity breach and not even know it's happening, while unscrupulous competitors or other bad actors steal their data or cause other problems. This situation is slowly changing, but not nearly as fast as the industry needs."
Luckily, many process engineers and other system integrators like Huff are awake and aware of the need for asset vulnerability assessments; thorough cybersecurity policies and procedures; implementing basic security measures such as network segmentation and traffic evaluation; and training staff and contractors to maintain secure behaviors. Even better, they're also beginning to share their cybersecurity know-how with each other, which can undoubtedly make them and any who pays attention more secure than going it alone. At last, almost everyone is talking about how to achieve and improve cybersecurity, and that's good news for them and everyone.
[sidebar id =1]"Even though cybersecurity can be a huge and costly problem, there are some pretty cool solutions out there like Secure-NOK's data table monitoring software that monitors for changes in the images of PLCs," adds Hoff. "I'm also encouraged because the U.S. Dept. of Homeland Security's Industrial Control Systems Cyber Emergency Response Team has been around for awhile, and it's putting out more and better cybersecurity updates, alerts and networking recommendations (Figure 1). Also, Control Systems Cyber Security Association International started about a year ago, and it already has chapters worldwide and reports it has 10,000 members. I was at its meeting in Detroit, and there were lots of OEMs sharing best practices."
Tom Lycans , senior consultant, Matrix Technologies Inc., a CSIA-certified system integrator in Maumee, Ohio, reports that, "Over the past two to four years, we're seeing more clients migrate toward more secure control systems that do what the ISA99 and IEC 62443 standards recommend, such as protecting manufacturing zones with firewalls and demilitarized zones (DMZ) between them. They're also using Microsoft's Active Directory Domain Services to manage users and groups with policies that restrict access, copy data and protect intellectual property. We've installed these secure controls at steel, pharmaceutical and food companies."
Greg Pfleghaar, department manager at Matrix, adds that, "More of our clients are also asking for us to proactively add firewalls, shutdown unnecessary functions and ports, and not allow publication of files into their systems."
Basics and buy-in...
One of several common threads that emerge as users share their cybersecurity efforts and procedures is: start by researching what equipment, assets and facilities they're running, how they're networked, how critical and potentially dangerous their applications could be, and what added protections they require.
"Our worldwide automation strategy at BASF begins with a pyramid of doing the basics before adding new functions and new technologies,” says Keith Dicharry, director of process control and automation for BASF in North America, who spoke at ABB Customer World 2017 in Houston. “The fundamental tasks at the base of that pyramid include safety and security. The first devices that did machine-to-machine communications were pneumatics, and the strategies used by today's controls aren't a lot different. The change now is that digital devices can be hacked into, and there are people trying to get in."
Dicharry cautions that cybersecurity projects need to avoid getting distracted by "shiny things disorder," which is adopting tablet PCs and smart phones before making sure they'll add value. He stresses they should also avoid "everything, everywhere syndrome," which seeks to make all data available, everywhere, all of the time, but often neglects to contextualize data, vet it, and get it in front of the right person at the right level for the best decisions.
In addition, Dicharry states it's important to beware of the two words "it's secure" because most static security measures will only be effective for a short time. "If they're not set up correctly, smart devices are very accessible, vulnerable and hackable," he explains. "Most vulnerability testing firms are very successful at breaking into process industry companies and applications, which are also affected by human factors, too." These include physical security breaches as well as purely cyber intrusions, he adds.
...at BASF
From a big picture view, cybersecurity at BASF is handled much like it is at other process industry firms: by IT at Layers 3 and 4 of the seven-layer Purdue Control Hierarchy Reference Model, which include manufacturing execution systems (MES) and enterprise resource planning (ERP) systems. Layers 1 and 2 are outside of IT's scope, and are typically handled by proprietary systems. "Layer 3 is a gray area, so we want some kind of cybersecurity middle ground from there on down. There are very diverse solutions below Layer 3, but we can use some IT methods," says Dicharry.
"We wanted a better plan, so we set up our Automation Security Team and a Global Technical Engineering Automation Security (GTEAS) team. We also got our IT security side involved to see what they could add to the process side. We found there were a lot of holes in that approach at first, so we also developed a combined operations technology (OT)/information technology (IT) SecurityOperationsCenter, which is up and running and meets monthly. "We've also been developing a cybersecurity solutions catalog, so when we find vulnerabilities happening, we can pull a solution from the catalog that includes input from other sites, or we can develop a new solution with help from partners like ABB." To help users monitor their network traffic, ABB also offers its Supros network management validation service.
Dicharry reports that one way he and his BASF colleagues get their company to pay closer attention to cybersecurity is by getting them to understand that it's very similar to process safety. "If people can understand that there's a problem, it's easier to justify funding for it," he says. "There's just no magic bullet. We can't show return on investment on preventing cyber attacks, but if we do a good business case and risk assessment for cybersecurity, we're usually able to get funding. We've even been able to add cybersecurity to BASF's overall automation roadmap, and we're benchmarking where BASF is on cybersecurity compared to how other companies are doing.
"Once a cybersecurity risk assessment is done, it goes to the Automation Security Steering Team, and then to senior management for funding. In our process, we don't try to use scare tactics. We've had to deal with some issues but have been able to keep them from impacting production thus far."
Dicharry adds there were some early disputes within BASF over cybersecurity policy. "For example, we wanted to allow ABB to have remote access to some equipment to assist troubleshooting, but ran into a brick wall with cybersecurity experts. “Now, they're getting more OK with managed remote access," he says. "My advice is be realistic, not pessimistic. We want to make chemicals and profits, and we have to use technology that's secure to do it."
Essential security steps
Once existing devices and networks are inventoried, and criticality and safety impacts are assessed, users and integrators report they can begin to draft security policies, procedures and technical requirements.
"Good cybersecurity design really needs an engineered approach, where you think through what's needed before anything is plugged in," says Dan McKarns, engineer at Matrix Technologies. "From the top down, we need to ask 'does this make sense?' Security should be integral to the overall system design, not something that is bolted on later as an afterthought. First, we need to take into account the overall organization of the system and its structural ability to support good controls. Then, we look at the interaction between different security domains, how devices are talking, how to segment the network, when to allow access, where we need security points, and when to inspect traffic between network devices."
Scott McNeil, senior network and security engineer at Global Process Automation (GPA), a CSIA-member system integrator in Wilmington, N.C., adds that, "Cybersecurity is a growing, living thing that isn't going to be perfect, but we must constantly strive to make it better. However, technology is changing so fast that, if anyone just pulls their head in like a turtle, they're going find they're a dinosaur and get bypassed, and they and their industry will have to struggle to catch up. IT has been involved in a cybersecurity arms race with hackers for 20 years, and industrial control systems (ICS) are now involved too because of all their connectivity."
Because cybersecurity on the plant floor is still in an early, wild-west phase that doesn't require participants to use any particular protections and leaves the choice up to them, McNeil reports that larger companies typically add some cybersecurity capabilities, while smaller ones do it less often. "Many users don't even have a stable industrial network, but that's what they need to run their DCSs and controls—and it's also what they need for cybersecurity, " he explains. "This begins by documenting the devices and equipment that need protection, which indicates how to stabilize their network and how to develop the right security posture. It's not rocket science, but standards like the National Institute of Standards and Technology's (NIST) "Guide to Industrial Control Systems (ICS) Security" can be a big help."
McNeil adds that GPA has encountered large, flat, vulnerable networks in oil and gas applications with no segmentation, where every device can see and talk to all the others, where all traffic is broadcast to everyone, and where there's lots of extra chatter, data collisions and instability. "Segmenting these networks by adopting a virtual local area network (VLAN) and Internet protocol (IP) strategy to reach it let's us get rid of a lot of chatter and gain network stability," he says. "It's most important to separate the plant-floor network from the corporate network, and use two opposite-facing firewalls to create a DMZ for any communications between them. This lets users receive historian data, antivirus upgrades, Windows patches and other updates as needed without compromising overall network security."
Patti's Hoff adds that one big and easily avoidable mistake is putting IT-style network switches on the plant floor in their default configuration, which can make them vulnerable because there's fewer defined restrictions about what devices they can communicate with. "You need to do a site assessment and configure each switch according to what that site needs," he says. "They need to be programmed to talk to this device but not that one, based on a study of the overall system. We've been doing this while applying Siemens' Scalance switches at a large automotive facility. This is just like doing a risk assessment (RA) for safety, and it always continues and needs monitoring, which is why we also have to dedicate resources to watching network traffic, too."
