Perception and the Cyber Security Challenge Face to Face, Roke Manor, 7th July 2017

This post was originally created for assessors, organisers, and participants of the challenge.  If you'd like to be sent an electronic copy containing full size images please contact us

Overview

Roke Manor Research Limited hosted a Cyber Security Challenge event on Friday 7th July 2017 in which a scenario was created for teams of participants to understand the vulnerabilities in a fictional company’s internet-of-things products.  In order to understand the events of the day, the Perception Cyber Security team were asked to deploy a Perception sensor to the challenge network to record all network activity that occurred during the challenge, both live as it happened as well as for later analysis.

The purpose of this document is to describe the activities seen by Perception throughout the course of the challenge, as a way of demonstrating the simplicity and coverage of a Perception deployment.

About Perception

System

Perception is a network security tool designed to give an analyst complete visibility of their network and potential threats that they may face.  Perception was initially designed by Roke Manor Research Limited (Roke) for the Defence Science and Technology Laboratory (Dstl), part of the UK Ministry of Defence (MOD), in order to detect anomalies on a network.  After successfully trialling the prototype systems, Perception was developed into a full product that combines multiple cutting edge technologies with the original anomaly detection system to provide one of the most advanced network security capabilities in the world.

Perception can be broken down into 3 distinct parts:

Data Collection

Using data collection technology initially developed by Roke for Lawful Intercept (LI) purposes for law enforcement agencies, Perception collects and analyses all network traffic at the core of the network at very high speeds.  This ensures the system has the best data pool to work from in order to make logical decisions later on.  Although an analyst is unlikely to pore over this low level information, this information is available to the user for analysis and incident response activities.

Behavioural Classification

By using Roke's expertise in cyber research for national security agencies worldwide, behavioural classifiers were developed that would understand the context of communications passing over the core of the network.  This is done by using a combination of anomaly detection, deep packet inspection, and database querying, rather than a single technology.  Looking at traffic behaviourally, rather than using signatures of known threats, is useful because it allows the system to identify threats without any prior knowledge of how they work.  The user is able to see a complete list of behaviours on the network in order to understand what may be threat like, indicative of misconfigurations, or indicative of vulnerabilities.

Artificial Intelligence

The final part of Perception is an Artificial Intelligence (AI) that constantly looks for correlations between the behaviours being stored on the system.  This AI is constantly being updated to mimic the activities of an analyst, in order to automatically and immediately identify links between multiple behaviours in order to detect vulnerabilities and threats.  This AI vastly reduces the time burden on analysts who would normally have to manually find linked behaviours, and allows Perception to alert with a very high detection rate and an incredibly low false alarm rate.

By combining these key technologies, Perception can rapidly draw a user's attention to indicators of threats, compromise, and vulnerabilities so that network security issues can be addressed before they become a serious problem.  The behavioural nature of the system allows Perception to detect zero-day threats without any prior knowledge of the malware, as well as detecting user error or malicious user behaviour that provide significant detection problems for firewalls and antivirus systems.  The ability for an analyst to identify misconfigurations or vulnerabilities represents a general theme within the network security industry to move towards a more proactive approach to the problem of protecting networks, closing vulnerabilities before they are exploited by an attacker, rather than just responding to threats as they happen. 

Perception sensors are easily deployed, consisting of a 1U rack mounted device.

Perception and the Cyber Security Challenge

The Challenge itself is one of many events set up by the Cyber Security Challenge UK organisation (www.cybersecuritychallenge.org.uk/), a not-for-profit organisation with the aim of bolstering the national pool of cyber skills.  As sponsors of the event, Roke agreed to host this particular event, testing 42 participants from around the UK.  Challengers were selected from a larger group of applicants who successfully completed some pre-event challenges, and none were currently working in the network security industry.

The Roke organisers of the Cyber Security Challenge Face to Face (F2F) event contacted the Perception team to discuss the use of Perception as a solution to be the “all seeing eye”, overseeing the challenge as it took place. With the high volume of hacking activities taking place on the day, it was vital for the assessors to have a tool that could quickly identify and hone in on participants actions.  The assessors were tasked with ensuring the rules of engagement were adhered to and the claimed courses of action could be validated, and Perception was used to carry out this task.

The Perception team were keen to exercise Perception on a network with such a high volume of potentially malicious activity.  They were also interested in better understanding which behaviours would be triggered where Internet of Things (IoT) devices were deployed.  Based on the brief provided by the Cyber Security Challenge team, the Perception team’s main objective was to be able to alert the assessors to any rule breaking as it happened and therefore demonstrate Perception's ability to proactively detect.  In addition to that the Perception team sought to provide detailed post-analysis of events carried out during the day, to provide the assessors with the necessary evidence to back up claims made by the participants.

Differences from a Real-World Deployment

The Cyber Security Challenge was an unusual deployment for Perception, which is typically deployed within the networks of commercial organisations.  Whereas normally Perception would be deployed to the core of a network with multiple normal users carrying out their normal day to day business, in this scenario it was deployed on a tiny network which hosted a large number of hackers, a number of IoT devices, no normal user traffic, and active malware.  Although there is value in noting the difference between this scenario and a standard Perception deployment, there are salient commonalities and threat scenarios that are present in both Perception's natural habitat on a network for a commercial network, and the Cyber Security Challenge F2F's infected, hacker-dense, IoT-focussed network.

Firstly, the activity of a large number of hackers allowed Perception to prove that it would handle detecting all of the activities of the attackers and active malware, rather than a single attacker or piece of malware.  Although Perception is well-suited to networks of all sizes, it is very seldom deployed on networks with the presence of multiple malicious actors simultaneously and this gave it the opportunity to demonstrate that even in extreme circumstances Perception could still handle the accurate detection of multiple threat sources.  In real networks there have been instances of an attacker infecting a high number of devices simultaneously in order to cause maximum damage or to hide their true intentions, and it is important to the Perception team as designers of network security systems to demonstrate they have a tool that can handle these types of scenarios.

Secondly, IoT devices are usually thought of as being used in homes, rather than businesses.  This is only partly true, a huge number of businesses deploy network attached devices such as smart TVs, IP cameras, and access control systems in their offices.  The management of these devices is usually seen as the responsibility of a facilities department within a business, which typically means they aren't subject to the same security and software update controls that would be enforced by an IT team.  Even amongst Perception’s current customers it has detected the use of IoT devices running old software that could be vulnerable, and as IoT devices become more and more mainstream, this is only going to become more of an issue.  It is a very common occurrence in today’s security landscape for an IoT device to be a first point of infection within a network due to their poor design, relative lack of security updates, and the inability to install anti-virus software on them.  The Cyber Security Challenge F2F event is a perfect opportunity to show Perception can identify these types of threats where other protections aren’t suitable.

The logistics of running a challenge of this type also raised some minor differences, for example the networks themselves were not connected to the internet, providing a safe environment for the event.  Participants were only allowed to use the provided Internet laptop for research on a separate Internet connected network outside of the challenge network.

About the Cyber Security Challenge Face to Face Event

Setup

The Cyber Security Challenge F2F event was a day-long event based around a smart home.  A fictional IoT device manufacturer, EKOR, had heard reports that some of its devices were less secure than initially thought.  During the course of the day somebody malicious would exploit a home network and was going to use these exploits to physically break into the home by hacking a smart lock on a front door.  The attacker would achieve this by exploiting a vulnerable server and then using a separate vulnerability in the update mechanism to deploy malware to the IoT devices in the victim’s home.

7 teams of 6 participants each were ‘hired’ by EKOR to try and find system vulnerabilities and give feedback to EKOR of what they should do to solve the issues, preferably before the attacker gains access to the home. The participants were briefed that EKOR suspected there were vulnerabilities in their products, but had no information on what activity was to happen on the day.  Their activity would include looking at the EKOR network and how the smart devices worked in order to gain an understanding of what the vulnerabilities may be. The teams were against the clock to get the information to EKOR as there was a set time for when the attacker was going to break into the home.

Perception Deployment on the Cyber Security Challenge Network

Each of the 7 teams had 6 laptops to work on (one for each participant) and a scaled down version of EKOR’s smart home products, a hub, a light, a door lock, and a camera.  All of these devices were connected to a switch specific to that team.  Finally, the teams were given an internet connected laptop which was separate from their switch so they could look up anything they needed to.  The seven team switches then fed into an 8th ‘core’ switch.

Simulated EKOR internal servers and other simulated external servers were also connected to each team switch to give the illusion of a real world network, as well as to facilitate the activity planned for the day.  This gave the teams a realistic environment to work with while ensuring isolation of all the teams.  Other than the separate internet connected laptops, the challenge was conducted on a standalone network with no connection to the Internet.  Any IP addresses/domain names/etc. used for the ‘external’ devices are purely fictional. 

The Perception sensor took a SPAN feed from the core switch, meaning it could monitor all activity on the network.  The Perception sensor then used a virtual private network (VPN) to communicate with the Perception Central Correlation Server (CCS) which aggregated behaviours and displayed them in the UI for the Perception team to view.  The CCS can be hosted locally or remotely, however in the interest of keeping the challenge network as simple as possible, it was decided to deploy remotely and communicate via a VPN in this instance.

A live stream of the Perception UI was in the lobby of the event location alongside the assessors, this allowed rapid communication between the Perception team and the assessors about breaches of the event’s rules and the teams’ progress during the day.

As it Happened

Morning

Stage 1

During the first stage, teams were asked to use provided tools and documentation to gain an understanding of the EKOR network. They needed to request access to certain compressed (.zip) files and packet capture (.pcap) files which gave vital information about their network. Using these files they should have gained a good understanding of the devices as well as how the network behaves. The packet captures provided were designed to give the teams an indication of which servers might be vulnerable.

Perception Analysis

During this stage Perception discovered data being transferred from EKOR servers to team’s devices as the teams downloaded the packet captures and the .zip files. By analysing this data the Perception team could see which teams were further ahead and which teams needed further guidance. The judges also used this information to understand who had broken rules of engagement by downloading information prior to being granted permission. Data was being transferred from EKOR servers to the teams via an unencrypted service, HTTP over port 80.  Since Perception captures a sample of the packets passing across the sensor it was possible for the analyst to view the actual file details and confirm their contents.

Figure 1: This screenshot of Perception’s UI shows .zip files and packet captures being downloaded by one of the teams

1) These micro-controls in the header provide a quick reference to the key metrics for the event such as source and destination of the data transfer, the number of sessions over which the transfer was made and the data volumes in both directions.  The button on the far right downloads the actual packet captures so they can be viewed in a packet analysis tool such as Wireshark.

2) This Data Transfer diagram shows the direction of the connection, the service used for the transfer (HTTP port 80) and the number of sessions used between the source machine (left green box) and the destination machine (right green box).  The larger orange bar shows the high volume of data downloaded relative to the low upload volume indicated by the thin blue bar.

3&4) These bar charts show the volumes and duration metrics of the transfer.  These charts are particularly useful when analysing data transfers over multiple sessions.

Stage 2

Teams were then given disk images that they could run, only some of which had been infected with malware. This should have given the teams some idea as to what the malware does as it becomes active on the network. The teams were then allowed access to EKOR’s software code base, allowing for manual code review to look for vulnerabilities in the systems. The malware would connect out to an external Command and Control (CnC) server to receive instructions on what to do.  Over a half-hour period the malware began to turn the lights of each team’s scaled-down EKOR products on and off.  This should have indicated to the teams that the malware was present on the network as well as indicating which devices were infected.

Perception Analysis

Once the malware became active on the network Perception saw connections to the CnC server. This allowed the Perception team to get an understanding as to which devices were infected.  On a real world network this information would be an indicator of compromise, enabling the analyst to gain an understanding of which other devices were connecting out to the malicious server, and therefore which devices had been infected.

Figure 2: This screenshot of Perception’s UI shows a behaviour that indicates an internal device has connected to an external device, in this case a compromised device connecting to the malicious CnC server.

1)  From these micro-controls in the top bar it is easy to identify the source and destination IP addresses, device hostnames, the service (port) being communicated with, and the number of other hosts talking to that same service.

2) This network diagram shows a source host (black circle) on the internal (trusted) network communicating with a destination host (red dot) on the external (untrusted) network.  This diagram also shows a number of other hosts on the internal network, (green dots) also communicating with this external device.  This is useful for quickly identifying which other devices have connected to this external host.

3) This summary information here identifies the key attributes for the main communication between the internal and external hosts, namely the IP addresses, hostnames, number of sessions and number of other hosts connected to the same destination.

Afternoon

Stage 3

The first task of the afternoon was to begin penetration testing. Penetration testing is the name given to the process of actively testing devices for potential vulnerabilities.  Teams were supplied with rules of engagement and were expected to ask for permission before actively communicating with the devices under assessment.  This is typical in a penetration test to ensure there is no unwanted impact on service.  Permission was granted, providing a narrow subnet of 10.31.0.0/26 to test against.  This stage consisted of a lot of information gathering from the teams using techniques such as port scans to identify active systems within that subnet and also what services they may be running.

Perception Analysis

From this stage Perception reported, somewhat unsurprisingly, a large number of scanning activities within the specified subnet.  In addition to this Perception was able to spot teams scanning wider than the subnet specified by EKOR, along with any teams scanning before EKOR had given permission for the penetration testing to begin.

Figure 3: This screenshot from the Perception UI shows a behaviour that indicates a port scan has occurred.

1) These micro-controls in the top bar show the key information about this behaviour, the source, number of destinations – separated by the defined network range, and data volumes.

2) This network diagram shows an internal host (black circle) communicating with one other internal host (green dot) on 503 unique sockets (number on the green line) using over 500 ports (number in the green dot).

3) This summary information shows the overall number of sessions generated by this host is 999, all of which were reset by the server.

4) This details table shows each scanned port as a separate row as the source device cycles through the available port range on the destination.  The analyst can easily use this table to identify any ports that elicit a response by sorting the table by TCP Flags B>A.

Stage 4

EKOR released more packet captures to participants that displayed activity from the malware allowing the teams to gain more information about the malware and the CnC server. Some of the teams may have already been aware that there were more systems in the upper parts of the subnet range from looking at some captures. Teams were expected at this point to request authorisation to start penetration testing on the wider subnet having discovered that there may be a vulnerable server outside of the allowed scanned subnet. Once requested, EKOR gave permission to scan the 10.31.0.0/25 subnet. If teams had not found these vulnerable devices then EKOR eventually requested for a wider subnet to be penetration tested. On the wider subnet there was a legacy server that had not been disconnected from the network when a replacement server had been commissioned. The legacy server contained some vulnerabilities that allowed it to be exploited, allowing an attacker to steal the password database for offline brute forcing.  EKOR had used the same administrator password on both servers so by gaining access to the legacy server, the attacker could use information learned to access the new server.

Perception Analysis

Perception observed the data transfer of the .pcap files being downloaded from EKOR’s file share at the point these were released by EKOR. Perception then raised similar events to the last stage indicating scanning activities but this time on the wider subnet. These events were used to verify with the judges if teams had prior permission to run the scans on the wider subnet.

End of the Challenge

The last stage of the challenge was for the participants to verbally present their findings to EKOR.  These would have included information about the vulnerable server, the malware deployed, and urgent remediation activity required to solve the issue.

Rule-breaking

Throughout the day monitoring by Perception was taking place to ensure that all teams followed the rules and also to help with the scoring of teams. Examples of some behaviours spotted included port scans before permission was given to the teams, scanning of systems that did not belong to EKOR, and not asking for passwords for downloaded files.

-          Each team was required to gain permission from EKOR before scanning any device on the network.  During the morning it was not expected for any team to be scanning the network, their aim was to gain information from the documentation provided by EKOR. Throughout the day there were a lot of scanning activities taking place that were captured by Perception. This allowed for checking that the teams generating these behaviours had asked for relevant permissions. Some teams had asked EKOR’s permission to run some scans but were only given permission to a small subnet of IP addresses. This meant Perception saw two types of rule breaking, scanning without permission and also scanning a wider subnet than permitted.

-          Teams were allowed to download documents from the EKOR file server to help them throughout the task, however these documents were password protected and access to them required asking EKOR for the passwords. This allowed the Perception team to check with the judges whether teams had asked for the passwords once they had downloaded the files. If these passwords had not been requested it may be assumed the teams used different means to open the document and thus broke the rules of engagement.

-          Some teams also began to scan a domain name service (DNS) server that did not belong to EKOR. This would have broken the penetration testing rules of engagement. Perception raised this event which was then forwarded to the judges giving them valuable information that they may not have had access to otherwise.  The participants of this event were not working full time in network security roles at the time, and perhaps would not have been used to the stringent rules network security professionals are subject to in the real world.  Actively trying to detect vulnerabilities in devices where the owner has not granted permission (such as this DNS server) is an offence under the Computer Misuse Act.

Figure 4: This screenshot from the Perception UI shows a behaviour that was generated when a team started port scanning an external DNS server

1) This behaviour is almost identical to the other port scan shown in Figure 3, however the IP address here shows that this port scan was carried out on a device outside of EKOR’s network range.

Actions of malicious third parties on the network

Behaviours from third party hosts were identified by Perception early on in the task. One of the behaviours included CnC connections from the infected IoT devices once the malware became active on the network. Perception raised events which indicated which IoT devices connected to the third party CnC server. These events from Perception showed that at least one device from each team connected out to the CnC server. If the participants had access to a Perception device they would have been able to verify instantly which devices were connecting to the CnC server and therefore understand which devices were compromised, substantially reducing the time taken to investigate the problem.  Likewise, if Perception was used as a vulnerability detector by EKOR, it is unlikely these issues would have been open for very long at all since Perception is designed to draw attention to vulnerabilities prior to them being breached.

Conclusion

The Cyber Security Challenge as a whole was a huge success.  The organisers were pleasantly surprised by the outstanding capabilities of the participants and the event as a whole represents a bright future for network security professionals within the UK.  The format of the event, one based around the growing threat of IoT devices was a welcome change to similar events held in the past and tested aspects of the participant’s capabilities that perhaps haven’t been scrutinised before.  Although some rules were broken along the way, the event gave the participants an opportunity to make these sorts of mistakes in a ‘safe’ environment while they hone their skills as security professionals.

Teams were scored accurately based on good behaviours shown and marked down where necessary when rules of engagement were broken. Perception assisted the judges in making these decisions by providing them with definitive proof of an activity that occurred and identifying the teams involved. In cases where teams denied any rule breaking, Perception was able to provide a record, often in the form of actual packets collected from the network, showing that they had done.  Perception observed a huge amount of behaviours on the network and correlated these behaviours into an actionable format to ensure the users of Perception could work efficiently. Perception performed very well in this environment due to its ability to begin identifying behaviours and generating alerts almost immediately with little or no configuration, this was vital given the short duration of the event.

Alex Collins, who helped organise the event for Roke, commented, “Perception provided excellent network visibility throughout Roke’s Cyber Security Challenge. Perception discovered the malware on the compromised devices and enabled us to quickly detect, investigate, and understand the activities of contestants throughout the day, as they tried to assess the security of our fictional Internet of Things product line and services.“

To learn more about Perception, please contact us.

Collated and written by:

James Crawford, Perception Analyst

Glynn Barrett, Perception Software Engineering Team Lead

Dan Driver, Head of Perception

 

The Perception team would like to extend their sincerest thanks to the Cyber Security Challenge for the event itself and the provision of assessors, as well as to Roke Manor Research Limited for putting the F2F event together, we know how much of an epic task this was.  They’d also like to give their utmost congratulations to all participants that took part in the event, their skillset was truly incredible, even more so given a relative lack of experience in the field, and it was an absolute pleasure to spy on you all for the day.