In this educational webinar led by the head of Microsoft’s data center security program, you’ll learn how today’s data centers are complex industrial-scale facilities sitting squarely at the intersection of operational technology (OT) and information technology (IT).

View the presentation PDF here


Webinar Transcript


Carol Auth:

Hello everyone, and welcome to today’s SANS webcast, Securing Building Management Systems and Data Centers with Continuous OT Security Monitoring, sponsored by CyberX. My name is Carol Auth of SANS. Today’s featured speakers are Imran Mohiuddin, Partner Program Manager at Microsoft, and Phil Neray, VP of IoT and Industrial Cybersecurity at CyberX, a Microsoft company. With that, I’d like to turn the webcast over to our presenters.


Phil Neray:

Thank you, Carol, and welcome, everyone, to our SANS webinar today. This is Phil Neray. I’m pleased to be joined today by Imran. I’m going to start with context-setting and then pass it over to Imran. I’m going to talk a little bit about what is OT security and how it’s different than IT security. Some of you may be new to OT security, so I’m going to talk about some of the differences, but also some of the similarities. I’m going to briefly introduce Azure Defender for IOT, which is the product that Microsoft launched a few weeks ago that has the CyberX technology integrated with it. Then we’re going to hand it over to Imran, who’s going to talk about how Microsoft protects its Azure data centers—not just from a technological point of view, but he’s also going to talk about some of the organizational challenges that he’s been addressing with his team, as well as incident response and removing silos.

So, let’s talk about OT security. This is a definition that Gartner uses, and the biggest difference between IT and OT security is that OT security is focused on protecting physical assets, like turbines, HVAC systems, and mixing tanks, in industries like manufacturing, energy generation, or smart buildings. Many of you may not think of buildings or data centers as being ICS environments, but Imran is going to talk about how they really are. The fact that OT is concerned with protecting physical assets brings a number of differences as well between IT and OT security. The primary one, which many of you may already be familiar with, is that OT security is focused primarily on safety and availability. If you’re running a factory or generating energy, the availability of those production facilities is critical to fulfilling a role not only in generating revenue, but also safety, as we know is a top priority in these organizations.

Certainly though, you’ve all heard about how the protocols that are used in OT are different than the ones used in IT. So, we’re talking about industrial protocols that might be proprietary or used only for certain manufacturers, like Rockwell Automation, Schneider Electric, and Yokogawa. The devices are embedded devices, but there are also legacy OS platforms used to monitor the processes and also to program these devices. Traditionally, these environments were air-gapped, but with digital transformation, that has certainly changed more and more. We see them being connected to the IT network, but also to the internet as well. For many years, there was less of a focus on OT security, because it was assumed that that air-gapping provided all of the security that was required. That has changed now, and there are many more pathways for adversaries to get into these networks.

The biggest difference is that whereas in IT security we’ve spent 15-20 years or so adding layers of telemetry and monitoring and control. In the OT environment, most SOC teams and most CISOs have little or no visibility into their OT risk. That translates directly into three types of business risks. The first is financial. We’ve all seen how destructive malware ransomware can shut down factories. We saw that a few years ago with WannaCry shutting down major automotive manufacturers, with NotPetya shutting down pharmaceutical companies and container companies, with LockerGoga shutting down an aluminum manufacturer last year, and Ekans, which is “snake” spelled backwards, a new type of malware that specifically searches for and shuts down ICS processes. The second type of risk is IP theft of trade secrets, such as information about proprietary manufacturing processes or formulas. An interesting example is one that was discovered by Microsoft security research, in which adversaries were compromising vulnerable IP connected devices that still had default credentials. The very first thing they did when they compromised those devices was they scanned the network to look and see what other systems they could access to get sensitive intellectual property. And the third, of course, is safety. There are many examples here: the Triton attack on a petrochemical facility, and the safety controllers in that facility, is a prime example. Another example would be a campaign that was discovered earlier this year, in which known vulnerabilities that were unpatched in building access control systems were being exploited. These are vulnerabilities that have a vulnerability score of 10/10; remote execution from an unauthenticated attacker would allow them to compromise these systems. We’ve also recently heard Gartner using a term called siegeware, in which you can imagine cybercriminals locking up a facility and keeping the occupants inside until the ransomware is paid.

Gartner has also recently written about something they call cyber-physical systems (CPS), and mentioning recent reports by the FBI and the NSA about attackers going after these systems. They’ve made some predictions that 1) CEOs will be personally liable for cyber-physical incidents, similar to the way they became personally liable for Sarbanes Oxley-related violations about 10 years ago, 2) that that will result in a large financial impact, and 3) that we should be focusing on operational resilience management (ORM) beyond what we have, as an industry, traditionally focused on, which is information-centric cyber security: protecting the data, either the security or the privacy of the data. What they’re suggesting now is that we should be focusing on cyber resilience from an operational point of view. The FBI report that they were referring to, or the CISO report, is this one came out just a few months ago in which CISA reported that adversaries were using many of the same tactics that we saw on the Triton cyber attack to compromise embedded devices in OT systems.

Then, they had three recommendations: 1) creating an accurate and detailed inventory and map of your OT network, 2) using the information about the asset inventory to prioritize, how to address the vulnerabilities and risks, such as unpatched systems, unauthorized connections between subnets, or perhaps unauthorized connections to the internet, and then 3) finally implementing continuous monitoring with anomaly detection. With respect to the IP theft are some interesting statistics here from the most recent data breach investigations report, in which they said that manufacturing is 8x more likely to be targeted with these types of attacks than any other vertical, and that more than one out of four of these breaches in manufacturing are motivated by cyber espionage, primarily from nation-states. And you can imagine, as we’ve seen the stories recently about pharmaceuticals being targeted for the research they’re doing on COVID-19.

So, let’s talk about Azure Defender for IoT. This is the product that Microsoft offers that integrates the agentless security that CyberX has developed since 2002, and it has a number of use cases. 1) Asset discovery: what devices do we have? How are they communicating? Obviously, if you don’t know what you have, you can’t protect it. But this information is also very useful in implementing zero-trust strategies, where you know how they’re communicating, and it makes it much easier to isolate them and segment them on their own networks. 2) How do you prioritize addressing vulnerabilities with those devices, for your crown jewel assets? Those are the assets and processes that, were they to be compromised, would cause major revenue impact or safety incidents. 3) Continuous monitoring—not just the monitoring, but also, how do you respond? How do you investigate? How do you do threat hunting? 4) Is not cyber-related, but it turns out to be huge value for our clients, which is they have the control engineers working in these plants, who have very little visibility themselves into what’s going on in these networks. So, when the device starts to malfunction or is misconfigured and is spewing packets all over the network and causing systems to go down, how do you look at that traffic and identify which device is the culprit? That turns out to be a huge benefit for the folks in the plants themselves. 5) In many organizations, traditionally there have been silos between IT and OT. So, how do you break down those silos? How do you communicate incidents from one team to another, in a common language, with a common understanding, and leverage all of the workflows and tools that we’ve built out over the years in our SOCs to handle these incidents? Certainly the workflows may be different, and they may need to be modified. For example, if an attacker is downloading control logic into your controllers, you need to investigate to see if it was a malicious upload or a legitimate one, but you want to be able to leverage the people in your SOC and train them on how handle these types of incidents in a different way, but using some of the same tools, the same SIEMs and SOAR tools that you already have—ticketing systems, for example.

So, to start contextually, how is the system deployed? It’s uses a technology called passive monitoring, also called network traffic analysis, in which a sensor that’s on-premises is connected to the network switch, grabs a copy of the traffic, uses specialized analytics—in our case, these are patented OT-aware behavioral analytics, specifically developed for OT environments. If you try to use behavioral analytics algorithms that were developed for IT networks in an OT environment, you’re not going to get optimal results. It’s going to be a much longer learning period, and you’re going to get more false positives and false negatives. CyberX is the only company in the industry that has developed a patented behavioral analytics approach specifically tuned to OT networks, using something called finite state machine modeling. So, from there, you can view the results of what has been analyzed in terms of assets, vulnerabilities, and threats. The solution will continue to be available 100% on-premises if you choose to, and we will also be making available Azure-connected solutions as well, and then integrating with what you already have in your SOC, whether that’s Azure Sentinel as your SIEM, or others—Splunk, QRadar, ServiceNow—are all available by out-of-the-box integration with this product. Of course, integrating with the SIEM is very important, because you want to see attacks that often cross between IT and OT or back. This is an example of the network map that the product will start to display within minutes of being connected to the SPAN port, so you can see that the devices are here. You can see the types of devices. It’s arranged in the Purdue model. You can double click into these devices, learn more about them. Who is the manufacturer? What protocols are they using? When were they last active? Then you can look at what unpatched vulnerabilities exist in these devices, what ports they’re speaking on, what the security score for each of them is. Then there’s an example of a secure alert here, where a firmware update might be legitimate or might be malicious, but you want to investigate. You want to know who to call in the factory or in the plants to find out who made this update to the firmware. In terms of how this fits into Microsoft’s overall security portfolio, there is a family of products called Azure Defender that are XDR extended detection and response solutions for various other components and part of Azure security center, and they’re integrated with Azure Sentinel as well. Azure Sentinel brings a number of capabilities to the industry: 1) it’s the first cloud native SIEM source, with all the benefits we’ve come to associate with cloud native architectures—simplicity, scalability, easy updates, continuous updates. We’re also doing a number of things to beef up the OT-specific capabilities of Sentinel: bringing in deep contextual information from Azure Defender for IoT—not just generic alerts, but also information about the devices that were involved, from OT-specific intelligence developed by Section 52, which formerly was the CyberX in-house research team focused on OT, malware, adversaries, and campaigns, and some OT-specific SOAR playbooks as well.

And to give you an idea here, this would be an example of looking at a Triton incident in Sentinel. You can see here that there was a device involved in updating the programming on that PLC, and you can click on it and get more information about that device. Again, this is the contextual information that was captured by Azure Defender for IoT now being displayed in Azure Sentinel, so to give you more context in investigating the incident.

Now I’d like to hand it off to Imran, who’s going to talk about how Microsoft secures its data centers. A.


Imran Mohiuddin:

Thank you, Phil. I’m just going to simplify this and give you guys a simple story. So, we are on a journey. By no means have we solved all the problems. You can never be 100% secure, but the OT plus IT plus now IoT space is very complex in nature, and we are on a journey. We are a lot better than where we were a couple of years back, and we are constantly improving our security posture. So with that, I’m going to give you some context of what data centers are and how you can relate them to industrial facilities. So, let me first talk about the scale. Microsoft’s cloud is global in nature. We serve a billion customers and more than 20 million businesses through our cloud, so it’s a very large scale problem that we are trying to solve. We take security and compliance very seriously. We have heavy investments in there. We have 90+ compliance offering support. That includes ITAR, HIPAA, GDPR, FedRAMP High, and things like that. We’ve invested about a billion dollars in our cybersecurity space, and our central SOCs process 6.5 trillion global signals. Let’s pause here and think about the scale that we’re dealing with. Now, let’s switch gears and talk a little bit about what data centers are. Data centers, the building blocks for the cloud, are nothing but industrial facilities. They’re comprised of electrical power monitoring systems, building management systems, HVAC, water-chilling systems, and fire suppression systems. Also, as you know, in the industrial control space, there’s a long lifespan of all these systems that you put in place. So, we are on an eighth generation. We have been building data centers for 31 years now, and we have multiple generations in production, because we have to support them.

Phil touched on it, but just to kind of ground us in the data center world threat landscape, crime syndicates are really targeting data centers because there is an activity increase in ICS, OT facilities. Ransomware is something that’s being hit by a lot of cloud hosters, and they are being impacted, which has been in the news. People realize now that adversaries realize that cloud operation is critical infrastructure, and they’re making it a priority to perform advanced threat operations over there. Also, what we are seeing in this world is a broader approach from adversaries—earlier, we saw localized attack vectors, and now we are seeing things like Ripple 20, which is targeting ICS, OT in a broader manner. Basically, Ripple 20 is a vulnerability found in a TCP IP library that’s used by many, many systems. So, there is an approach change by the adversaries to impact the systems in a broader manner.

Now, what is our team’s approach? As I said, we are on a journey, and stepping back a couple of years ago, we decided, how do we go solve this problem? There are multiple issues over here—as we talked about, multiple form factors, cultural problems, and things like that. So, we established four work streams in the team. The very first one is called Manage & Protect. Microsoft runs something called the “get secure and stay secure” campaign which had been running in IT security across the organization and across the hierarchies. Our patching vulnerability status by services is reported out to the leadership, so we kind of adopted that practice of Manage & Protect for the data center world.

The second thing we realized is that we have some visibility into the data centers, but not full visibility into the data center stack. This has been achieved with leveraging things like Azure Defender for IoT, Azure Sentinel, and integrating them together—and not only just stopping there, we needed a good IR response for the OT world too. So, we had a central SOC, which was responding to IT security-related events. We needed to expand that, and we have integrated with our central SOC, as well, for OT capabilities. Third, as many of you know, in the ICS world, there are many personnel who are full-time employees of your organization, vendors, part-time employees who are going in and out of data centers or remotely connecting to the data centers. If they bring their own devices, things like USBs, you are exposed to a lot of risks. So, we kind of worked on that too, and we have a work stream going on on that front. Finally, the last work stream we call Cybersecurity Services, also known as security by design. We have made heavy investments in doing security activities jointly with our partners to improve the culture. This includes security assessments, best practices, reference architecture, blueprints, and things like that. We do that in partnership with not only our own data centers, but there are a lot of data centers that are leased, so we incorporate our relationships with the lessors and do this activity in a joint manner.

Now, let’s give you a perspective of how we have increased visibility into the data centers, using our security monitoring system and integrating with incident response system and Microsoft. The very first thing was collecting the data. We realized very quickly that we don’t want a partial picture. We want to correlate and get a full picture of what’s going on in the data center. So, we collect data from various systems, whether it be Microsoft managed security devices, physical access, biometrics, patching, domain controllers, 80 firewall logs, and then to get the ICS network data, we leverage Azure Defender for IoT, previously known as CyberX. So even before the Microsoft acquisition, we have been using CyberX, and this is where we get a lot of security insights and and inventory that we take and plug into our Azure Sentinel system.

So, data is collected from various sources, brought into Azure Sentinel, and this is where we perform our OT and IT playbooks. We perform our alerting, we have some detection mechanisms, we have some ML that we apply, which allow us to reduce false positives and then also correlate and make our alerts richer. And then this is what we integrate with our incident management system and our central SOC. All this information is forwarded over, and then we have a good feedback loop coming back into our correlation and detection system from our incidents, so that we can improve our alerting capabilities. Again, this has three different silos, but using Azure Defender for IoT and Azure Sentinel, we have brought these three different silos together, democratizing the information across many, many groups in Microsoft. So, multiple teams in Microsoft now have access to this information in the context of an event or an incident, by adhering to privacy and compliance.

Switching the gears a little bit, we talked about the technology part, but over here, to be very candid, in the earlier days when we were approaching the problem, we were not empathizing with the OT persona as much. We just felt naturally buyer and tech savvy, you know, “These are basic things. Why don’t they think about the world in the way IT security sees it?” We didn’t realize a simple patching could cause a disruption and then bring the systems down in production and it could lead to safety implications, availability implications. So, we spend a lot of time to deeply understand the OT persona.

So our team shadowed the OT activity by making site visits, bridging the gap with them, and establishing a good relationship. Then we established a common vocabulary for when we talk to each other, which really helped. And then we made them feel that their subject matter expertise is really important. And then they are our customers and we are trying to solve the problem for them. We also acknowledged and appreciated their priorities: importance of life, importance of safety and availability. I think those were our big learnings, and then incorporating that in our daily activity just built a bridge a lot better. So they had preconceived feelings about how the IT security approach is: if they come, they would cause disruption, they’re just a different team, and there would be audits and escalations. And this had happened in the past too.

So now what we do is we give the information in their hands first. It’s not like, “Hey, you know, look, leadership. This is what this team’s not doing well.” Rather, “Hey, we found a problem. Let’s solve it together.” So that’s the approach we take now. And then with these tools, Azure Defender for IoT or end-to-end security monitoring system, Azure Sentinel, we are giving them resources and data to proactively plan for worst day activity. The net net is that we are taking a proactive approach versus reactive approach with our OT partner team. Again, double-clicking a little bit on the approach, persona empathy was really important. Growth mindset is a key principle at Microsoft, that you have to constantly learn. So focus on understanding the OT plus IT persona together and what’s going on in their environment by both the teams—that really helped.

I talked about site visits and shadowing, doing two-way trainings, and setting up websites. I’ll give you an example of that. We have set up a dedicated portal. What should you do when things happen? If you feel there is a security incident, who should you reach out to? Simple things like having aliases that you can make a request to, or identify people or groups that you can reach out to really help. Proactive versus reactive approach—we talked about it. Instead of just doing this red team activity and sharing it with the leadership, we do that in a joint manner by keeping them in the loop—the right people, not everyone—and then we do a lot of security assessments together. We have also implemented a security assurance process to help the teams make a request for a security-related items, toolings, and services.

We have built multiple tools to manage the asset inventory and the end-to-end security monitoring and response systems, which are really helping. We have identified what success should look like. What should a blueprint in a typical data center look like? So doing that assessment and doing that comparison really helps. Finally, something that you cannot ignore and that goes a long way is bridging that gap and bridging that relationship through informal means. Right now we have COVID, but we still do virtual connects and things like that. We started doing lunch socials, weekly happy hours, and monthly connects.

So this is an example of setting up a dedicated site for our team who builds data centers to reach out for security-related events. Who do they reach out to, and who are the people they can contact? If they have a pen test request, who should they reach out to? This really helped and kind of streamline communication between the two groups. The other thing that we did, as I mentioned, is we standardized the communication and the vocabulary, not just with the language and terminology, but also with metrics. So the Manage & Protect workstream covers the metrics, and we have definition of saying, okay, when we say: you have code integrity metrics, what does it mean? Git, what does it mean? Who do you contact? Who owns that metric? And what’s the definition of the metric? And we provide a tool which shows your security posture across your services in one place. So it says, here’s how my vulnerabilities are. This is how I’m doing on TLS and things like that. And this rules up, it’s not just specific to the team in an org.

So if you see, I can search by organization, I can search by service, I can search by people, and this gives a rolled up view. So this information is given in the hands of the actual operators who can look at this information and act on it in proactive manner. This information is looked at by individuals to drive their action items on a on daily and weekly basis. On a monthly basis, leadership reviews where we are with our security posture and how we are improving on our action items. This is just an example of how we collaborated with multiple teams. We talked about central SOC, we talked about our central security team plus the data center builders team. We came together and we said, “Okay, what should happen when we have an incident? How do we come together, and what’s our standard IR process and set of playbooks?” This is just one example, and then we have created workstreams and we have co-defined and documented this, so people know what happens. This type of proactive behavior really helps. So now, overall, we talked about how we have increased visibility into the data centers and how are we helping improve the culture again. We’re on a journey, making a lot of improvements and heading in the right direction, but there’s still a lot more work to do. Leveraging Azure Defender for IoT and Azure Sentinel, what are we getting? One of the things that we are getting is improved, critical environment inventory. Earlier, we used to have static inventory. We had something, but it was still. It was just hard to keep up with. Now, with Azure Defender for IoT (CyberX), we can get real-time inventory understanding of what PLCs are being operated by what HMIs and how they are laid out in the network. With this, we can do proper risk analysis. We are also able to leverage Azure Sentinel and Azure Defender for IoT correlate to security-significant events. As I talked about, across multiple sources, we have already implemented detections for things like known malware, botnet, and command and control traffic. We are also leveraging a newly launched a feature around user behavior analytics, UEBA by Sentinel, to do insider threat detection. As as many of you may know, ICS networks are very interesting. As Phil touched on earlier, we thought we had segregated them, but not really. Things have changed. We have gotten a lot of disruption from connectivity means, commoditization, and how our corporate networks are porous with these networks. So our personas straddle between these two networks, right? That’s their journey. People go in, they download architectural designs and related documentation from SharePoint, and then they go and operate on the actual ICS OT network to change the settings, or use a facility operation engineering workstation to deploy a ladder logic, or something like that.

So if you don’t track the user profile from an end-to-end manner, you won’t be able to understand insider threat properly and understand the user behavior in an effective manner. So that’s where we are. We are maturing to implement UEB analytics across the user journeys, across the persona journeys. Then, as I mentioned earlier, our response playbooks were just IT-centric. We have now added OT plus IT related items, and then we’ve also implemented some logic apps, capabilities to integrate with our SOC for automation purposes. Finally, this is one area we are maturing on: once we added Azure Defender for IoT, we got a lot of alarms and a lot of events, and then we got on this journey to reduce the alarms and find needles in the haystack, which we are able to do with the capabilities that are provided by both Azure Defender for IoT and Azure Sentinel. This is something we are constantly improving so that we detect only the right anomalous behavior while reducing the noise.

There is a lot of work happening. We are a good showcase of the technology components that Phil talked about, but we are not there yet. We are on a journey, and not because we have gaps, but the space is evolving. As you all know, every day there is a new innovation coming up. The cost improvements is pressuring the system. So we go with a healthy dose of humility. We are building the data centers of different form factors. You may have seen the videos or the news on the underwater data center. We are building small data centers, big data centers, and then we are also, like many companies, investing in the 5G space and things like that.

So your security posture needs to be constantly evolved and improved with changing times. This is where we are, and we are constantly learning and evolving. We have multiple generations, as I mentioned earlier, and we are building many more generations. That’s our biggest learning: you have to have that growth mindset. Second, not everything we do in IT security can blindly be applied to OT security. This was our biggest learning. As I mentioned earlier, if you try to patch an OT system, there could be huge repercussions. You may just bring down your operations, causing safety risks. The liberties that you have in the IT world cannot be just blindly applied on the OT world, so that empathy is super important. The third learning is that we cannot take everything in IT security and apply it to OT security, blindly, but there are a lot of patterns in the IT world that can be applied to OT. And there has to be a model where we can take our assets and leverage them with context of OT. So we talked about Azure Sentinel. It was a typical scene, but what if it has OT capabilities built in? Azure Defender for IoT is another example. This is where we are making heavy investments as Microsoft to appreciate what the need is in the OT space, hence the acquisition and how it’s getting integrated. As Phil touched on, with the overall Microsoft stack, we have the Microsoft Threat Intel Center, MSTIC, that is being integrated and has been expanded with OT capabilities. CDOC is our Cyber Defense Operations Center, central SOC, which is now being added with OT playbooks. So yes, do not blindly apply IT security, but at the same time, do not throw away IT security best practices—mature them and boost them with OT capabilities.

That’s the biggest learning we had. We talked about addressing the culture gap. Again, this is also a journey. It’s a dial. You have to constantly improve. It’s not that, “Hey, our culture is perfect.” So you just have to live the principles and the guiding tenants and improve the relationship with the teams constantly. So I think that’s something that’s a constant practice. Fifth item is being proactive rather than reactive, which we touched on a couple of times. Then one final thing is we cannot do this alone, so we are leveraging partnerships. We have broader partnerships with ICS cybersecurity teams. Coming together as a community and helping the community improve their OT plus IT security posture is the way to go. So that’s a big realization that Microsoft had.


Phil Neray:

Well, thank you, Imran. Before we go to any of the questions, I just want to point out some other sources of information. If you want to learn more, there’s a page on the Azure site about Azure Defender for IoT. You can also view an on-demand session from our recent conference, Microsoft Ignite, where you’ll see a demo, specifically talking about the Triton attack, and it will also show the integration with Azure Sentinel. You can learn more about the architecture on the CyberX website. We recently held a series of roundtables with folks like Imran to talk about how to bring IT and OT together. We also had some folks from Novartis talk about how you do zero trust for OT networks. And there are some other areas in there that you might find interesting.

Okay, so there’s the question from someone about Azure Sentinel, and he’s asking if you have both IT and OT signals mixed together, how do you separate them so that the IT infrastructure team can focus on the IT signals and another team can manage OT? Imran, what are your thoughts on that?


Imran Mohiuddin:

Good question. So we do not mix IT and OT signals without context. Our OT and IT signals are mixed together by the hierarchy, so what we do is we identify a concept of a facility or a data center, and then say, these are IT devices joining to OT. If we try to just look at IT in a silo, in a facility, then we’ll miss the complete picture. So our goal is to put all this information together and understand what’s happening in an ICS facility from an end-to-end manner and identify anomalies. However, for these services, they have a different stream coming into Sentinel, which do not belong to a facility, and it’s not mixed together with the OT landscape. OT plus IT is encapsulated with a concept of a facility and an environment.


Phil Neray:

That’s great. Thanks. Also, in Sentinel, the alerts are separated by which platform supplied the alerts. For example, I know many of our customers know which alerts came from the OT security system and which alerts came from the IT security system. They can separate them that way. Then you can also put in place automated actions, such as using ServiceNow to assign a ticket to a specific group based on where the alert originated.

I have a question here from someone about noting that many OT devices are still working on old operating systems. In fact, in CyberX’s 2020 ICS Global Security Report, we found that 71% of all facilities were still running older and unsupported versions of Windows. So what do you do? You can’t necessarily upgrade them, because that would require revalidating the applications and retesting. And if everything’s working, nobody wants to upgrade them. So what we have found is that if you implement continuous monitoring as a compensating control, even if you haven’t upgraded those older operating systems, you will immediately know when one of them is being attacked and then you can quickly take action. Most cybersecurity folks today recognize you’re not going to prevent intrusions. The trick is how you detect behavior that would indicate an intrusion, so you can very quickly mitigate that attack before it blows up your plant or shuts down your plants or causes any safety incidents.

A question here: does CyberX support the SaaS model? The answer is yes. Many organizations still want to run it completely on-premises, and we will continue to offer it that way, but we will soon be offering an Azure-connected SaaS model for Azure Defender for IoT that will allow you to benefit from all of the cloud-related benefits of SaaS: automated updates, scalability, and simplicity.

Next question: can you please describe the main features of the sensors? So, as you saw in that slide, the sensors can be either physical or virtual. Some clients prefer a physical sensor, because it’s easier to manage from the point of view that they know it’s an appliance sitting in Iraq in their data center, but others prefer the benefits of virtual, which obviously has other simplicity benefits. Often the key issue is you have a virtual infrastructure in your OT network, and if you don’t have a virtual infrastructure in your OT network, it’ll be difficult to connect that virtual appliance to the OT network switch that it needs to be connected to. So sometimes that becomes the key way of deciding if you’re going to use a virtual or a physical appliance.

Another a question here: many of the OT devices need administrative access for day-to-day tasks, so how do you isolate the network? I’m not sure if I really understand this question, but it turns out that one of the key attack vectors for the adversaries is remote access from the IT network to the OT network, leveraging, for example, stolen credentials. So an adversary manages to get into your IT network, maybe through a phishing attack that steals legitimate credentials, giving the adversary remote access to the OT network. They can then get into the OT network and, using those credentials, have administrative access to your critical systems. This is a technique that’s been used way back in the first Ukrainian grid attack, and it’s been used many times since. So it’s another example of why you need behavioral analytics to detect this type of anomalous activity, and you can’t rely on static IOCs, because they’re not necessarily using malware. They might be using living-off-the-land tactics using RDP followed by PowerShell to get to your systems. You need to be able to look for suspicious or unauthorized behavior. You can’t rely on signatures, because they’re not necessarily using known malware to do it.

There’s one last question about building automation, and Imran, maybe you could say a bit more about when you say building automation, what systems does that cover, and how does that relate to ICS security?


Imran Mohiuddin:

Building automation systems for us include elevator services, physical access to doors, and things like that. That’s how we see building automation systems.


Phil Neray:

Yeah, and for example, we have another client that is securing skyscrapers in New York using this technology, and it’s the same issue. You’ve got PLCs, you’ve got any HMIs, you’ve got engineering workstations—they’re all being used to control, as Imran said, the elevators, the HVAC systems, and the building access control. So you’ll see all of the same types of devices in buildings as you do in a factory or an energy utility, maybe slightly different protocols—you might be seeing more BACnet, for example—but it’s the same basic idea.


Imran Mohiuddin:

Yeah—HVACs, lighting, access control, etc.


Phil Neray:

And to answer the question about which protocols are being monitored, virtually all industrial protocols are being monitored, as well as the standard IT protocols that you would expect like, HTTP and HTTPS, SNB, FTP and RDP. But over the years, since CyberX was founded in 2013, we’ve encountered many different types of protocols, both proprietary protocols the specialized protocols. So over the years, we’ve developed the ability to sniff these protocols and analyze them for the information that we’re looking for on assets, vulnerabilities, and threats.

With that, I think we’ll conclude the webinar. Back to you, Carol.


Carol Auth:

Alright. Well, thank you so much, Imran and Phil, for your great presentation and to CyberX for sponsoring this webcast, which helps bring this content to the SANS community. To our audience, we greatly appreciate you listening in. For a schedule of all upcoming and archived SANS webcasts, including this one, please visit Until next time, take care and hope to have you back again for the next SANS webcast.