Listen to our online panel discussion, where we joined forces with Adani Energy Business, Essity AB, and MundiPharma to discuss how to ensure cyber resilience and production uptime in the face of limited staff availability.

During this critical time, it has never been more important to ensure the continuous operation of your IoT/OT networks and facilities. This webinar covers how to:

  • Secure your critical networks from adversaries during periods of higher risk
  • Accelerate incident response by leveraging automated workflows (SIEMs, SOAR, firewalls, NAC, etc.)
  • Meet customer demand for essential goods

 

Our Panel of Experts

Ashtad Engineer, Head of Technology, Adani Energy Business
Gareth Stewart, Head of IT Security & Strategy, MundiPharma
Henrik Perrson, Information Security Controller (IT & OT), Essity AB

 

 

Webinar Transcript

Phil Neray:

Good morning and good afternoon everyone. My name is Phil Neray. I’m with CyberX, I’m the VP of IoT & Industrial Cybersecurity. Thank you for joining us today – we have nearly 300 registrations, so obviously this is a topic that a lot of people are interested in, and we have a great panel of experts to talk about it. You’re here to participate in a round table discussion with a group of OT security experts from various companies. Today’s panel includes Gareth Stewart, he’s the Head of IT Security & Strategy for Mundipharma, a pharmaceutical company with several thousand employees and several billion dollars in revenue. They specialize in medicine for respiratory illnesses, oncology, diabetes, and other illnesses. So, he’s the Head of IT Security & Strategy, and before, he was the Enterprise Architect for Mundipharma. Thank you for joining us today, Gareth.

We also have Henrik Perrson from Essity AB, a $100B+ company with nearly 50,000 employees specializing in hygiene and health with products including tissue paper (and I won’t make any jokes about how that is critical infrastructure today), compression products, orthopedics, and others. He previously worked at ABB, so he is acquainted with OT and automation technologies. He’s also a GICSP and certified in IEC 62443. We’re also joined by Ashtad Engineer, he’s Head of Technology for Adani Energy, India’s largest private power company with $25B in revenue and part of one of the top five conglomerates in India. His company manufactures electricity, and are involved in other businesses such as renewable energy. Adani are also involved in other businesses such as ports, agricultural logistics, and real estate. Before joining Adani, Ashtad worked at Schneider Electric (formerly Invensys) and as a program engineer at Rockwell Automation. So he, like the other two panelists, is very familiar with OT. My name is Phil Neray; I’ll be guiding us through this process. We want it to be informal, casual, more of a conversation. We didn’t ask anybody to prepare any PowerPoint slides. What we think people are looking for is a conversation about best practices, what works, what works less – and I encourage you to ask questions in the chat window and we’ll try to get to as many of them as we can. A few pictures just to give you an idea from their website – so this is Mundipharma, something that Essity posted on their website/on their LinkedIn a few days ago, and then Adani posted this on the left here to talk about their situation and how they’re addressing it.

So some of the questions we’re gonna go through today, I’m going to show them all and then we’ll sort of work backwards, but we’re going to start with some questions that are more about OT and IT, the convergence of OT and IT, and the implications of that convergence for security. So, everyone talks about OT/IT convergence, but people rarely talk about what that implies for security. And then we’re going to talk about a few questions relative to the current situation – how companies are keeping their plants running. All three of the plants that we’re talking about for the industries here today are definitely important, and more than ever, in keeping them running – how to handle some of the remote access situations that are coming up more now, and some other questions about prioritizing OT security, prioritizing security in general, and how companies are prioritizing the projects they’re going to work on. So let’s start with the question about bridging the gap between OT and IT security. I’m wondering if one of you, Gareth, Henrik, or Ashtad, would like to start with an answer to that question, please?

Gareth Stewart:

So, I think the main way that I approach the integration of OT and IT, especially with people, is I was a bit lucky where I had done enterprise architecture in my company before, and the main thing I did was a couple of networking projects with the OT folks that really opened me up to knowing who they are, knowing what they have to deal with on a daily basis, and then how can I help them. Then, when I moved over to a security role, I was already well-known to them, and they were also well-known to me. So there’s a bit more of an open door than other people might experience. But the main thing is to being in position to help somebody. That’s what will get you a good reputation – to help someone with a problem, whatever it is. If you’re an IT security person, you should leverage your other IT functions to go into leverage and help them with a problem if they’ve got one, or bring to bear some of the enterprise skills that the manufacturing team may not have just because they’re used to working in a very different way. That was my approach, anyway.

Henrik Perrson:

I could follow up on that and say from my perspective, this is very important. We say IT and expect everybody to know what IT means, and then we say OT and we expect everybody to know what OT means. And one thing that I’ve was very clear about is what operational technology stands for, so when we talk about it, people actually know the definition. Be very clear when communicating, both with IT about what OT is, and vice versa. Otherwise you can never determine the boundary between them, because they are more interlinked and intertwined than ever before.

Phil Neray:

And how do you and your company draw that boundary?

Henrik Perrson:

Essity is a large corporation. We have an IT organization and also a global manufacturing organization. That is a boundary from a corporate point of view. But then, we have other boundaries from a network point of view, an integration point of view, function, safety, all that. But what I would say is, if we look at it security, traditional IT security and information security, it’s all about governance. You should have your standards, etc. If you would take all those standards and apply them just straight off towards operational technology, there might be a slight mismatch between those areas. So it’s kind of about: we have our methodology, how do we adapt it? And also then with the statement – accept the similarities, but respect the differences between IT and OT. And I think that is the key, but we need to establish the term “operational technology” in order to do that.

Phil Neray:

Great, thanks. Ashtad, what are your thoughts?

Ashtad Engineer:

It’s a fantastic opportunity for IT to actually to understand the business. And to the point that there’s a difference, because at the end of the day, it’s network. It’s TCP IP networks. It’s just that the protocols running on TCP/IP are a real timeness, if you will. What we’ve been able to do is have the IT guys understand the process, sit with the process engineers and really start understanding. That is where really the appreciation comes about, you know – what IT is, what technology is, and what OT is. It’s a fantastic opportunity for the IT guys, too. I would really understand the business, which they do, and really understand a multilayered approach rather than the traditional enterprise approach. That’s what we’ve be able to do, is have the security guys and IT guys go to the plants, sit with the operators, understand stuff and how it’s architected. I think that becomes a very good enabler for us.

Phil Neray:

It’s a great point, Ashtad. One of the things we talk about with our clients is about protecting the crown jewels. We recognize that you can’t patch everything, but you need to protect the most important assets. And the way to find out what are the most important assets is to talk to the business as you were just talking about, to understand what production lines generate the most revenue or which production lines, if they were to be compromised, might cause a major safety incident. So yeah, you can only do that by talking to the business.

Ashtad Engineer:

And talking and taking that extra effort, too, for the OT guys. And mainly, the OT guys are control engineers. If you really look at it, if you go to a site, it’s a control engineer who’s actually looking after your DCS, your PLC, or SCADA with a bit of IT background, and the IT guys coming down from, “Hey, I’ve got a firewall switch, blah, blah, blah.” But, at the end of the day, it’s the understanding of both these worlds, which needs to converge, from an understanding of the process. And I think that is where it’s very important for each one of them to understand each other’s background, where they come from. And the best thing is to just have them sit together and talk it out and understand.

Phil Neray:

That’s great. That leads me to another question that we’ve been asked many times, which is: how do you help your IT security team understand the differences with OT security? It’s kind of the other way around – do you hire OT people to come work in the SOC for a couple of months? Do you send your security folks into the plants for a couple of months? What have you found works best there?

Ashtad Engineer:

For us, the best is to just go and live the daily life of a controls engineer down in the plant. That’s the best way you can actually understand the process. Because again, making sense of events coming from OT and IT, and bringing that convergence – the people have to understand, is it actually an anomaly, or is it actually a process of separatist? Okay. I think that you can only come if you are in the plant and at the site. We really advocate when we get inside security professionals and be able to go live, breathe, and eat for at least two weeks, three weeks at the plant and just understand. So that’s what we’ve been able to do.

Phil Neray:

That’s very cool. Anybody else?

Gareth Stewart:

Yeah, I mean for us, most of our IT folks have had a tour of the plant, whether they’re security or not. They will go through a half-day tour of what the plant is – we only have one. But, they will walk through and they’ll get an understanding of exactly what the process looks like. Just like Ashtad was saying before about understanding the process. So, the manufacturing process will then be at least reasonably well-known by any individual who’s walked through and can see how it all works, from the very beginning of the supply chain to once the thing goes out the door. And that’s a big thing. As I said before, we did some projects: we integrated a lot of our engineers into the OT space just by having them work together and making sure we understood in IT what are the issues in the OT space, to understand what the challenges are. As Ashtad was saying, this are very different challenges. So yeah, that’s how we did it.

Henrik Perrson:

We’re not as fortunate to have only one site where we can actually get the kind of dig down – we have 18 manufacturing sites around the world – but the big thing is to understand the differences between IT personnel or IT technicians that are located in pure business and sales offices, compared to the IT technician that is located on a manufacturing site. I would say the biggest benefit that we are doing is from within our audit program. For example, it’s when we do operational technology security audits and instead of only assessing availability, integrity, and confidentiality, as soon as you step into the OT world, you add safety, and talk about that and involve the local IT teams and the local OT teams over a table session doing that risk assessment together. It kind of goes with the entire data flow then from the sensors to the MES systems and to centralized systems as long as both parties are involved, both IT and OT. I cannot send central resources to each and every site to have a walk-around – of course, we have multiple business and functions as well. We have hygiene and health, we have medical, we have others. It’s very dependent on the situation, but also to respect the differences that we have.

Phil Neray:

Yeah, that’s great. I know that some organizations have IT working at the plant-level and others don’t. And one of the things I heard from the CISO of a major car manufacturer who had IT folks in the plant was that they immediately saw the benefits of having more visibility into what was going on in the network that they didn’t have before, and they also found that visibility could help the plant to be more efficient. It wasn’t just about cyber, but by having more visibility they were able to find equipment issues and misconfigured equipment faster. So, if you have the luxury of having IT people in the plant, definitely they will help you bridge that gap, because they’re trusted by the folks in the plant.

Henrik Perrson:

I would agree, but I would also not stretch it only to the internal people of our organization, so to say, because I see a big part that machine vendors, and maybe the old-school vendors, they still design systems as they should not be connected to anything. And I hate it when people say, “Yes, we should have automated asset inventory for a system.” Yes – but stop designing that system as a black box then. If it doesn’t have to be a black box, we need routing, we need those things in place in order to be very dynamic and aligned with digitalization, and Industry 4.0, and all those buzzwords as well. I see a good opportunity to improve, but maybe even before the machine hits the floor.

Phil Neray:

So, you’re talking about the automation vendors still thinking that they’re living in an air-gapped world where everything is proprietary. Is that what you’re referring to?

Henrik Perrson:

Could be. I’m not throwing everybody over that line, but yes, there are some. And, of course, as a corporation and company, you have to put your demands in place on vendors and understand that you can challenge them.

Ashtad Engineer:

Yes. And to Henrik’s really good point, how do you build cybersecurity by design? Because you can end up in a zoo. Each and every vendor has a different style of deploying projects. For example, you can have a flat IP structure. And how do you even get the purchasing guys to understand that sometimes when you do a shoot-out between two or three vendors, it’s not about the cost? You have this whole cybersecurity angle by design, which is very important for your purchasing departments to understand. So guidelines by design to, in Henrik’s case, machine OEMs, for us, big equipment – for us, we’ve come out with standard guidelines and we’ve said, “Hey, SCADA providers, automation providers, this is our multilayer-security approach. Now, your equipment and your automation gear has to be a good candidate of the equal system, which we have defined.” That becomes a very important aspect. The automation vendors don’t listen to that. They have their own way, and then come and try to sell some of their services. No. As a customer, I know exactly what I require. Understanding and listening to the customer is going to be very important for automation vendors. This is one of the most important points of OT security.

Phil Neray:

Interesting. Gareth, any thoughts?

Gareth Stewart:

Yeah, in healthcare we have all kinds of issues with suppliers. So, as the other guys are saying, there is no real standardization in that industry for security. There are the basics that we may expect to have in place that are nowhere to be seen. We have to work quite hard to even get systems patched if it’s Windows 10 or something, it’s so ridiculous, it’s like, well, why don’t you just do it anyway? Once you deploy, you can’t touch it. That’s it. There’s nothing you can do about it. So, there’s a lot of engagement with vendors going on as well, because that is quite a prevalent mindset, that security is just not really thought of. Because we’re just manufacturing stuff, we don’t need to worry about security. That’s not the case anymore when we have so much integration. And that’s the key difference now than 20 years ago, that integration is much more important now than it ever has been. So, we need to get reports back to essential restocking system to understand if there’s enough supply for the markers we’re going to send – do we need to make more, do we need to make less? The way to get that is to get data out of that manufacturing system or that manufacturing area, into the IT area where it can then be seen by someone to plan demand, for example. That’s becoming critically important.

Phil Neray:

We do this Global Risk Report every year where we look at anonymized vulnerability data that we capture from production networks, and we’ve found that more than half of the sites we analyzed aren’t running antivirus or aren’t running antivirus that automatically updates itself. And we thought that that sort of resistance by the OT manufacturers to installing antivirus had gone away. What has been your experience with respect to getting antivirus or anti-malware on the OT equipment where maybe 10 years ago they said you can’t touch it at all? Are they still saying that?

Gareth Stewart:

Anti what? [laughs] That’s pretty much how it is, it’s still very much the same. I mean, we still have equipment that is 30 years old that’s still running and no one’s gonna pull out a 30-year-old Unix controller and put in some Windows 10 thing to spend $10 million when something I have still works. No one’s going to do that. And vendors are still not supporting – they support us putting antivirus on stuff, they just don’t do it.

Henrik Perrson:

I fully agree in some areas. Then, we have other areas. We are more manufacturing in terms of tissue and DCS system is the term – big systems containing virtualization, Windows servers, etc. There I see the vendors such as ABB, Siemens – they are more mature. They at least provide you with alternatives. You can have an antivirus, but it needs to be an approved and tested antivirus, and you need to get the antivirus definitions from us, through a service, etc. They are providing alternatives, but it still comes down to how good those alternatives are when you cannot securely connect the system to an enterprise network or securely connect it to the internet where it needs to go to get updates. I’m not saying we should connect our industrial systems directly to the internet, but via controlled means and controlled integrations, we should be able to push and deploy antivirus updates, etc., to even our critical infrastructure. So, there is some mixed maturity, sure. I don’t agree that the resistance is the OT guys. I think the resistance is that it’s hard to achieve – it requires alignment with the vendor, alignment with the automation engineer, alignment with the infrastructure, and then alignment with IT, because IT needs to acknowledge that they are not using the IT-preferred antivirus. It’s a long chain of discussion if we cannot collaborate in good ways – so, it’s hard to implement and then maybe it’s easier to say “no antivirus.”

Phil Neray:

What types of compensating controls would you put in place if you can’t put antivirus?

Henrik Perrson:

Everything is risk-based, and I cannot tell what mitigating controls in details we would apply in Essity. But if you have a risk based approach and you, for example, look to the standard that is called 62443, if you determine a certain level of security or safety that you should have, you can apply the controls that are relevant for your function. And if your system is not capable of antivirus, then pick an option. But then I’ll also acknowledge that you have then kind of agreed to a risk. Everybody knows it, it’s no secret, it’s an industry best practice to do a network segmentation, for example. So, why not do that – but ensure your network segmentation includes next gen functionality? So you actually almost have an antivirus but on a network level instead. So things like that are mitigating controls. Whitelistening is another alternative as well, but I’ve not seen that being widely used.

Phil Neray:

But you were just now talking about zero trust as another alternative or a multilayered approach. And Gareth, you were talking to me the other day about network segmentation and what it was like before and what it’s like now, do you want to say a few words about your network segmentation projects?

Gareth Stewart:

We’ve done a lot of work around IT-to-OT segmentation. That’s kind of the first step that we went through. There’s that kind of boundary on the edge of each of the networks. So that middle part of that boundary was the first thing that we attacked while we were looking to procure a secure platform for manufacturing. And now that we have deployed the CyberX platform, we can then understand more about what’s underneath the network, which before was just – if you had a box and filled it in black, that’s what we saw before. Now we see a lot more detail than we ever have before. Even detailed machines that people didn’t even know it was still turned on. The OT guy said, “Oh geez, we gotta turn that off and throw it away, we thought it was already done.” There’s a lot of detail that we have and we can now go through and do that segmentation of each line, and the clustering of the manufacturing systems that need to talk to each other, and then segment them off from the ones that don’t. And that’s something that we’re still working on. We developed an architectural design at the very start as a basic blueprint of what we’re going to build, now we’re just implementing that as we go. I mean, it’s not 100% fixed; it can change, but we’re now putting in place, and having one plan is easier than having eighty. For me it’s much more simple than it is for Henrik or Ashtad. And hopefully the next 18 months we’ll be segmented. But there are still devices we can’t see.

Ashtad Engineer:

It is absolutely important, Gareth, and that would be the whole zero trust. And for the older technologies, I’m sure there are Windows 7 and Windows XP still running in production. Having the security around it from zero trust becomes very important – again, link it back to risk-based, and of course when the business has the money to upgrade the systems, that’s when they take it in that upgrade cycle. But that wasn’t as key in terms of getting the segmentation.

Phil Neray:

I think when we were talking a few days ago, I asked which projects were being prioritized now in the current situation where you might have fewer staff available and network segmentation. I think all three of you said that those projects were still high on the list. How do you handle the antivirus issue with your automation vendors?

Ashtad Engineer:

We are actually in the process of handling this right now. We have run tests on different SCADA stuff on really understanding this whole agent-based approach. We have taken a route where we are really seeing the traditional – you know, Symantec and Norton, which the automation vendors give us the patches for – but how could we get away from that and use a single agent system? And I think that is where the innovation is required because if you really see, you have different agents on the machines doing different things and different functionalities, and that is the convergence I would like to see in the innovation coming up with the security vendors.

So, yes, we’ve got programs where they give us patches, patches go out, we’ve got a multi-automation system vendor. But we have in our OT program, which is a three, four year program, is to really then to take the knob where ABB, Siemens and Schneider come to us. Not in different ways, but in something of a common way on how they would handle security. I’ve been discussing this at the world economic forum where I represent Adani in the electricity-cybersecurity value chain, and I have senior people from all these companies as part of the group. This is the topic of discussion which we do on a monthly basis. That is where, coming back to my point about, it’s just, “what about antivirus?” It’s more than that. It’s a layered approach to security.

Phil Neray:

Very cool. Thanks. Let’s talk a little bit about how you’re adjusting your strategies and your tactics in the current situation. Let’s start with the business level first. What are each of your organizations doing to ensure your plants keep running if people can not actually come to the plant? What are you doing to keep the plants running?

Henrik Perrson:

I would say that this is, on a corporate level, being handled by many teams throughout the company. I would kind of address this from my point of view, and also on the infosec edge, if I may, but I would say that really quick it was established that travel restrictions were going to be a thing, and we would not be able to travel to sites, even within countries. We are a hygiene and health company, so we know about that – good hygiene is always promoted. That is our nature here at Essity. But the big part that we were doing on manufacturing sites, if we narrowed the scope there was: when people were ordered to work from home, they kind of classified white colors and blue colors into two different areas to determine the critical functions that actually need to be on-site. I’m talking about operators that are not able to operate at a remote distance while the automation engineers might be able to operate via secure remote measures. That was one way that they separated production-critical services or personnel from business people and narrowed the scope there. Another thing that has been communicated is – let’s see if I get the term correct – less-impact production. Machines and sites or whatnot have been dedicated to run a certain product, for example, for a longer period of time to kind of minimize those risk moments of changeovers, etc., when key players might be quarantined at home. There’s many ways that it has been addressed, and it shows, of course, and unfortunately in times like this, not only with Essity, but friends of mine at other companies, they have challenges that the sites that are not allowed to be as digital as other sites because of restrictions or poor security are now the ones in the most trouble. I mean security and digitalization go hand-in-hand. And if you have good security and good digitalization in place, you’re better equipped for – well, the scenario we currently are in. That’s really the quick go-through of things

Phil Neray:

Henrik, your last comment is very interesting and not something that a lot of folks thought of in the past as a benefit of digitalization, and, as you said, the security that you need to have a company digitalization because of the higher degree of connectedness that it entails. How are you ensuring that all of these folks coming in remotely are doing it in a secure way, and that that you don’t have adversaries trying to get in at the same time?

Henrik Perrson:

I’m certified within the standard 62443 and IT/OT segmentation is a thing, and there’s good IT/OT segmentation and there is bad IT/OT segmentation. We wish to utilize all the next gen features, all the firewalls we need – we wish to utilize all those features so that even if something would get into a certain area of our networks, it will be detained within that area. So of course, it depends. It always depends, but we have concepts that we prefer on how to do remote access, and they are of course deemed to be at an adequate risk level. But we should know and this is what we’ve seen in the last couple of weeks. Like I said, I have a large forum at ABB as well. Vendors of machines are pushing out quick-and-dirties, or remote solutions, etc. In these times, we need more remote solutions. “Here, you have a 4G modem and an industrial router connected, we can support you.” Things like that might get implemented in folks’ environments, and it’s something that should be under consideration. So, increased monitoring of the boundaries that you might have as well, if you have a well-defined IT/OT boundary – I feel that is very important in the current situation.

Phil Neray:

Increased monitoring, that’s good. Gareth or Ashtad, do you want to take the question? Well, it’s a two-part question. How are you keeping your plants running, or in the case of Ashtad, your generation plants running, and then how are you ensuring that people are accessing those networks in a secure way from a remote location?

Ashtad Engineer:

We have all our data on the cloud. So the plant historians, we did that, we didn’t have any compromised systems. So we have a very secure way to get on the cloud. We do all our analytics and predictive stuff on the cloud. What we did as part of this was really look at maintenance schedules for three to four months and really link it back to our risk-based approach from an asset perspective which would require physical travel to the site. And by that, we’ve been able to use the calendar to avoid disrupting operations. Fortunately for us, our capital maintenance cycle comes up every other, from a September/October perspective. So we are bailed out from our biggest maintenance shutdown, but the fuel one, we’ve already taken a risk-based approach for the next three, four months.

Phil Neray:

That’s great. Thanks. And how are you handling the secure remote access part?

Ashtad Engineer:

We have systems which we have deployed from the vendor giving us remote access. If you look at different businesses, our renewables business has remote access and we already have security built around that. And that has more to do with our panels and trackers. But what we’ve seen with our gear, there is not much we could do from a security, from a remote access perspective, for them to do maintenance – when we do our risk-based maintenance for us, which doesn’t expose us too much to the outside, especially to the language.

Phil Neray:

Got it. Thanks. Gareth, what are your thoughts on keeping time monitoring and then ensuring you have secure remote access?

Gareth Stewart:

While a lot of the measures to not travel heavily were introduced in the UK about middle of March, our plant put heavy restrictions on people going in-and-out at the end of February. So they were actually two or three weeks ahead of global guidelines. So we weren’t able to just walk into that building and walk around. We were just completely restricted. There were very heavy restrictions even then, which obviously necessitates the heavy remote access that they used. So, we’ve got very well-defined remote access paths into the network. There are two or three different ways that it could happen, but it could only happen that way. If someone tries to connect to one of those remote access motors, it’ll be detected, and the IT engineers won’t allow it. They worked with us to come up with a number of ways to connect and that’s the only way that even they will support their vendors connecting. It’s very well-defined as part of our reference design for the plant. That’s one part we implemented and it’s working really well. That’s an important piece we recognized early on.

Phil Neray:

Let’s talk about more about your other security strategies. When now you have fewer folks available to be in your SOC, triaging the alerts and responding, what are some strategies that you’ve found have been effective from a security point of view?

Gareth Stewart:

The dance for us is actually really easy. We’re mainly based in Cambridge, UK. Our SOC is in Chennai, India, so they’ve always been remote to us anyway. And a lot of our IT people are used to working remotely anyway; we have a fairly global team with folks in Asia Pacific and in the US as well. So for us it was a bit simpler than most other companies, which is lucky for us. So we didn’t have that much of a leap in adjustment to make.

Phil Neray:

Great. Anybody else?

Ashtad Engineer:

Yeah. Same here. Our SOC is monitored by our infrastructure partner, and what we are seeing is really integrating the IT and OT logs together. That, for us, has started becoming very important. That visibility today was just, “Hey OTs here and ITs here; we recognize that there’s already a project underway to do get that single visibility into one song.”

Henrik Perrson:

I will also answer, and I would say “business as usual.” We’re a global, distributed team. I would say there has been an increase of phishing attempts and COVID-19 assigned tasks, etc. Of course, we are highlighting those and taking the necessary measures, but otherwise, for our security defenses and security teams, it’s business as usual.

Phil Neray:

That’s great. And also going back to what Ashtad said about unifying IT and OT security at the SIM, this is what I’ve found is fairly common across our clients, whether you’re using Splunk or QRadar or ArcSight or Logarithm or RSA Netwitness. The idea that you forward the alerts into the SIM, that you train your SOC personnel and the differences of handling alerts that might be OT-related. For example, you might get an OT alert that says somebody sent the PLC stop command. Your average SOC analyst is not going to know if that’s good or bad or who to call to verify whether it was a valid or malicious command, but they have all kinds of other workflows for addressing lateral movement and understanding how attackers move through your network that are just as applicable, I’ve found, for OT as they are for IT. Any thoughts there?

Gareth Stewart:

We integrate our logs as well from OT and IT, just like Ashtad does – that’s a must. We, of course, deployed CyberX, but that was a day-one thing that we had to do, and we knew that we had to correlate all that stuff to try and form a picture of any bad-acting that was going on.

Phil Neray:

And then how do you train your SOC analysts to deal with alerts that might be unique?

Gareth Stewart:

We have a workflow that would direct anything that a SOC analyst doesn’t know towards the lead OT engineer. And if they didn’t know that, it would come to either me or the head of operational security. The workflow is really important for us to get this right. That’s taken a while. It’s not a quick process; it takes time for them to learn what all those alerts mean to then dig into them and investigate. After about nine months, it’s getting better, but it’s an ongoing process.

Phil Neray:

How about the rest of you on that question? If you’re unifying your alerts at the SIM in your SOC, but yet there are differences between OT-related alerts and IT security-related alerts, how are you modifying your workflows so people know what to do when they see a PLC stop or a firmware update?

Henrik Perrson:

I would say one thing to this entire discussion: if you’re going to monitor and react to downloads, uploads, and set points values, you kind of have a whole set of other foundations you need to have in place before. That is the last piece of the puzzle, I would say, depending on which angle you’re tackling and what business you are in. What I would think is that this type of approach is very tight, that you have a good network architecture, and that you have a good understanding of within which zones you are allowed to do certain things. Because then, it’s easier for you if you can tie this to, as Gareth said earlier, to a concept. If we have a concept of how we wish our operational technology network in our IT networks and the dependency and integrations between them should look, maybe it would be a better idea to start focusing on the boundary and then dig down into the different zones of the parameters within the system itself, and the so-called zones and conduits within it. Because if you try to monitor all the set point changes, all the OPC alarm and events, etc., it will be overwhelming. So maybe it’s discard that zone, discard that area for now, but information through and from that area is something we will look at. So, don’t start with it all. Start with well-handled boundaries instead, and then work your way through it. And I think that is what Gareth said as well – it’s an ongoing process. This is not something you activate and are done with the next day.

Ashtad Engineer:

And you learn from the previous process and workflow to do it better the next time. I think that is what it really boils down to. For us, it’s a control room where you have fleet production of our entire assets. So, we actually have infrastructure and cyber as part of the control room when you’re doing this kind of triaging. To us, rather than talking on the phone, we thought, why didn’t we get the person out there so he can really talk to the operator and ask whether it’s an anomaly, or that’s somebody doing something? We found that coming out of a use case where we had to get on a call and talk to the security guy, that’s kind of the process improvement you do.

Phil Neray:

Yeah, that’s great, Ashtad. Something I’ve found talking to our clients is when these projects for OT security are just getting started, even knowing who to speak to, who to call is a first step.

Ashtad Engineer:

And I think all of us would agree that senior management starts becoming very important, especially at the OT layer, because the risk to the business is the whole risk, because ultimately, this is risk management. And I think the education – not just of the IT and OT guys, it’s the education even for the C-level team – which I think is a major KRA for the security organization and the IT organization coming together and doing that. So stakeholder communication at the CEO level becomes very important in an event that you’ve been attacked or there is a threat of loss. As Gareth said, it’s a work in progress. You’re not going to get it right the first time. But I think that mission becomes very important to see your risks and communicate them to your senior management.

Phil Neray:

That’s interesting, because that’s something else I’ve heard from our clients – sometimes we get the question: is it top down or bottoms up? And what I’ve learned from talking to our clients is that it really needs to be both. Ashtad, what I think you were just talking about was the top down. You need to get the people that run the business.

Ashtad Engineer:

You need to. In my experience I have faced it, but the senior management has to have a half-day workshop, about the business, the operations. It’s the COO. I think the whole management team has to really understand that is where the communication aspect becomes very important.

Phil Neray:

From a bottoms up point of view, what are some of the strategies you find work best?

Ashtad Engineer:

Training, education. When the IT guy comes and says, “Yeah, I want to do a VAPT,” – you should be in a position to say, “No, you’re not going to do a PT on a running system. You can do it when we have a shutdown.” Things and subtleties things like this have to be – and that can only come when you educate when you talk and when you communicate.

Phil Neray:

I’ve also heard many OT teams have been burned by penetration tests or patches that were forced on them and reboots that brought down production, and production is the number one no-no, obviously, in our business.

Ashtad Engineer:

Correct. There is always a human behavioral aspect of the OT person that sees security as an overhead. “Security isn’t letting me do this, they’re not letting me do that.” The only way you could really get across that human behavior as a barrier is to come to a point and come to a consensus where the education happens just not on one side, but the education happens on both of the levels.

Phil Neray:

That’s great. Anybody else want to comment on this really strategic topic?

Henrik Perrson:

I wouldn’t comment on the entire strategic concept, but I would like to comment that it’s kind of interesting that you can, in some cases, present security and align it with the benefits of digitalization. I usually say that if you want to do digitalization, then security is a foundation or a compliment to that – and vice versa. By enhancing security you can stay creative, enable modern ways of working, remote access, and all that. It’s always interesting to understand or get the feedback when people say, “Could we do that?” Yes, you can do all those fancy things if we get security in place. Then you can interact more, you can do more integrations, and you can have more scalable solutions. So, thrive on education, yes, but thrive on the business need as well, which is a lot around digitalization and modern ways of working – more than ever at these times that we’re in right now.

Phil Neray:

We’re getting to the top of the hour. I want to thank everybody for their participation, especially my panelists. You guys were awesome. I really enjoyed the conversation. Have a great rest of your day and keep your plants running and secure. Thank you very much.