Skip to main content
Guidance

Cyber Assessment Framework

The CAF is a collection of cyber security guidance for organisations that play a vital role in the day-to-day life of the UK, with a focus on essential functions.

Page 17 of 25

Principle C1 Security monitoring

iStock.com/varijanta

Capabilities exist to ensure security defences remain effective and to detect cyber security events and incidents adversely affecting, or with the potential to adversely affect, essential function(s).




Log collection and aggregation 

Having the correct visibility of your systems and users is critical to detect potentially adverse activity. It is possible to detect adverse activity at an early stage by collecting, aggregating and analysing the appropriate logs. Below is a non-exhaustive list of some log sources and what you may wish to consider looking for:

  • Web site traffic going to and from the internet. As a minimum this should include domain names, URL’s and IP addresses, but if appropriate, stretch to the full header information. Monitoring this traffic will help to identify threats, for example malware beaconing, unauthorised access attempts and even potential DoS attacks.

  • Email traffic. As a minimum, the metadata about what is sent and received, but if it is possible to capture both headers and content, then consider doing so. Phishing attacks, delivered over email, often tempt the user to click links, download attachments or perform other unwanted actions. Getting visibility of information about emails such as links, sender address reputation and attachments in combination with web traffic helps detection and subsequent analysis.

  • IP connections between your network and others, including the internet. It is useful to monitor for unusually high outbound data volumes especially to unknown or untrusted IP addresses. 

  • IP connections between security zones in networks. As a minimum, capturing 5-tuple metadata from critical zone boundaries such as the security zone interfaces are important. This IP traffic is likely to contain evidence of adverse activity within your network and information systems and so detection strategies should be in place to identify these indicators of a compromise.

  • Host-based activity. A host-based monitoring system can detect unauthorised activity on computer systems themselves (e.g. unusual or unauthorised activity by users or software systems), which might evade detection systems focused on network interfaces. 

  • Deviations from normal interaction with systems (e.g. user activity outside normal working hours) and unusual patterns of network traffic (e.g. unexpectedly high traffic volumes, or traffic of an unexpected type etc). 

Your log collection should capture the activities of users both regular users and privileged users, at the various system layers (e.g application and operating systems). This helps to identify adverse activity.  

Duration and level of logging is a business decision, balancing risk, impact of compromised systems, regulatory requirements etc with the ability to detect potential adverse activity as well as retrospectively query data during (and after) an incident. Consider any legal data protection laws you may need to adhere to on the collected information. 

The audit and log data should be held securely with access controls that limit access to authorised personnel and systems with a business need and is appropriately secured (e.g  isolated from less trusted domains. This is important as it will reduce the likelihood of an attacker from accessing, deleting or modifying logs.  

Your organisation's asset management processes should ensure knowledge of network and information systems is sufficiently detailed and accurate to quickly and efficiently trace observed events to their sources.  

Monitoring and analysis tools

 The collected logs should be compared against signatures, and baselines of normal activity to be able to detect anomalous behaviour indicative of adverse activity. 

You should choose appropriate tools to help analyse and correlate differently structured and normalised network datasets, to identify and investigate potential security incidents. These tools should be chosen to optimally scale to and use the types of log data you expect to analyse and the workflows you have in place to analyse, triage and investigate. Your monitoring and detection personnel should receive the appropriate training to use these tools and data. 

Consider the flexibility of the tools used, as you do not want to preclude your analysts from proactively finding unknown threats (as described in  CAF Principle C2). Avoid purchasing black box tools that do not allow flexible querying or provide results without showing the corresponding rationale.

Understanding System’s Behaviour & Threat intelligence

Having the appropriate understanding of the behaviour of your network and information systems that support your essential functions and relevant as well as accurate threat intelligence are key requirements for any security monitoring capability. 

Being able to determine what ‘normal’ or ‘expected’ looks like is a fundamental step towards identifying anomalies indicative of adverse activity. Identifying behaviour such as which networks, systems and users should be exchanging traffic, how much should be exchanged and the activities of each of these over a period of time can help you build a picture of what is ‘normal’ or ‘expected’.  

The following areas can assist you with identifying ‘normal’ or ‘expected’ behaviour - Asset Management, identified and understood data flows, known roles and responsibilities, approved and understood processes and procedures, identifying baselines of ‘normal’ or ‘expected’ behaviour as well as the knowledge monitoring and detection personnel hold of the essential functions(s). 

Just because something appears abnormal doesn’t mean this is a sign of adverse activity. Your understanding of systems behaviour should be coupled with your understanding of the threat (Threat Intelligence), even if this is from a very high level to determine whether the identified event warrants further investigation. 

Threat intelligence can come in many formats, volumes and quality. It can be collected from open discussion forums, trusted relationships, paid-for contracts with threat intelligence companies or generated internally.  

Threat intelligence can be either automated feeds that describe Indicators of Compromise or more descriptive human readable reports. You will likely need to consume various types (e.g strategic, operational and tactical) of both internally and externally produced threat intelligence. 

We would recommend that if choosing automated threat intelligence feeds you favour quality over quantity (false positives can be costly for analyst’s time) and ensure the feeds can be automatically ingested by your chosen analysis platform and made available to the necessary users as appropriate. 

Governance, roles and workflows

Your monitoring and detection teams should comprise roles and responsibilities that cover both security and resilience related monitoring. 

The size and structure of these teams will vary between organisations, but should include people who know the network, its hardware and software, the types of data and activities that they process and produce. The team should also include personnel, who can work with threat intelligence to identify, investigate and triage security events and managers who understand the organisation's business and are able to assess the significance of security events in terms of their potential to cause harm, such as disrupting operations or leaking sensitive corporate or personal data.  

Your monitoring capability should work seamlessly with Incident Management (see CAF  Objective D), knowing when and how to alert on or escalate events and how to share the right sort of information with the necessary individuals and teams. Monitoring and Incident Management may even comprise some of the same personnel.

Regular review and update 

Your monitoring strategy and capability should evolve with your business requirements, networks and systems. That is, as the system develops (e.g. new systems, networks or software versions), the monitoring capability is updated in order to ensure that the network and information systems supporting your essential functions(s) are covered. Your capabilities should also evolve to keep up with changes in the threats you need to mitigate.  

Your tools should be configurable and adjustable to handle new datasets and your monitoring and detection personnel should be able to work with these changes. New systems should be designed to produce log data that allows the appropriate level of monitoring before they are made operational.  

C1.a Sources and Tools for Logging and Monitoring

The data sources that you include in your logging and monitoring allow for timely identification of events which might adversely affect the resiliency of network and information system(s) supporting the operation of your essential function(s).

Not achievedPartially achievedAchieved
At least one of the following statements is true:All the following statements are true:All the following statements are true:

Data relating to the security and operation of network and information systems supporting your essential function(s) is not collected.

You are not able to audit the activities of users and systems in relation to network and information systems supporting your essential function(s).

You do not monitor traffic crossing your network boundary.

Log data cannot be synchronised using an accurate common time source.

Logs are stored in locations where they are not readily available to authorised users and systems.

Your monitoring tools cannot be configured to make use of new log streams as they come online.

Your monitoring tools are only able to make use of a fraction of the log data being collected.

You do not understand where log data is stored or how long it should be stored for.

You have no way of ensuring log data is being captured as expected and available when needed.

Data relating to the security and operation of some areas of network and information systems supporting your essential function(s) is collected but coverage is not comprehensive.

Some user and system monitoring is done, but not covering a fully agreed list of suspicious or undesirable behaviour.

You monitor traffic crossing your network boundary (including IP address connections as a minimum).

Some but not all log datasets can be easily queried with search tools to aid in investigations.

Your monitoring tools work with most log data, with some configuration.

Your monitoring tools can make use of log data that would capture all common threats.

You ensure log data is available for analysis when needed.

Monitoring is based on a thorough understanding of network and information systems supporting your essential function(s), techniques used by threat actors, and awareness of what logging and monitoring is required to detect events and incidents that could affect the operation of your essential function(s). 

Your monitoring data provides enough detail to promptly and reliably detect security events, incidents and support investigations. This is reviewed regularly and after a significant security event.

Extensive monitoring of user and system activity in relation to network and information systems that support your essential function(s) enables you to promptly detect policy violations, suspicious or undesirable user and system behaviour, deviations from normal / routine behaviour or abnormalities indicative of adverse activity.

Your logging and monitoring capability includes host-based and network monitoring.

All new network and information systems supporting your essential function(s) are considered as potential logging and monitoring data sources to maintain a comprehensive monitoring capability.

Log datasets are synchronised including using an accurate common time source so that separate datasets can be correlated in appropriate ways.

You enrich log data with other network and information systems data to provide a more comprehensive picture of actions and behaviours.

Your monitoring tools make use of log data to pinpoint activity.

You regularly review the data sources and tools included in your logging and monitoring strategy to ensure it remains effective.

C1.b Securing Logs 

You hold log data securely and grant appropriate user and system access only to accounts with a business need. Log data is held for a suitable retention period, after which it is deleted.

Not achievedPartially achievedAchieved
At least one of the following is true:All the following statements are true:All the following statements are true:

It is possible for log data to be easily edited or deleted by unauthorised users or malicious attackers.

There is no controlled list of the users and systems that can view and query log data.

There is no monitoring of the access to log data.

There are no policies for accessing to log data.

Only authorised users and systems can access log data.

There is some monitoring of access to log data (e.g. copying, deleting or modification, or even viewing).

You have defined and implemented retention periods for log data.

You have given legitimate reasons for accessing log data in your policies.

Appropriate access to log data is limited to those users and systems with a business need.

The logging architecture has mechanisms, policies, processes and procedures to ensure that it can protect itself from threats comparable to those that it is trying to identify. This includes protecting the function itself and the data within it.

Log data analysis and normalisation is only performed on copies of the log data keeping the master copy unaltered.

All actions involving log data (e.g. copying, deleting, modification, or even viewing) can be traced back to a unique user or system.

The integrity of log data is protected, verified and any modification, including deletion, is detected and attributed.

C1.c Generating Alerts

Evidence of potential security incidents contained in your monitoring data is reliably identified and where appropriate triggers alerts.

Not achievedPartially achievedAchieved
At least one of the following is true:All the following statements are true:All the following statements are true:

You do not apply updates to your detection security technologies in a timely way, after receiving them (e.g. AV signature updates, other threat signatures or Indicators of Compromise (IoCs)).

Security alerts relating to network and information systems supporting your essential function(s) are not prioritised.

The enrichment of security alerts within network and information systems supporting your essential function(s) cannot be performed.

You do not confidently detect the presence of IoCs on network and information systems supporting your essential function(s), such as known malicious command and control signatures (e.g. because applying the indicator is difficult or your log data is not sufficiently detailed).

You do not monitor for user or system abnormalities indicative of adverse activity.

Logs are monitored infrequently.

You easily detect the presence of Indicators of Compromise (IoCs) on network and information systems supporting your essential function(s), such as known malicious command and control signatures.

You apply some updates, new signatures and IoCs in a timely way.

Security alerts relating to network and information systems that support your essential function(s) are prioritised.

The enrichment of alerts within network and information systems supporting your essential function(s) is performed but not as part of the original alert.

Detections and alerting rely on off the shelf tooling without customisation or users reporting events and potential incidents.

There is a documented and shared process for all users who support the operation of the essential function to report events and potential security incidents.

Where appropriate, detections and alerting result in automated actions being taken. (e.g. malware identified by AV is quarantined).

You monitor on an irregular basis for user or system abnormalities indicative of adverse activity.

Logs are monitored at regular intervals.

 You easily detect the presence of Indicators of Compromise (IoCs) on network and information systems supporting your essential function(s), such as known malicious command and control signatures, as well as abnormalities or behaviours indicative of adverse activity.

You apply all updates, new signatures and IoCs promptly.

Security alerts relating to all network and information systems supporting your essential function(s) are prioritised and this information is used to support incident management.

Alerts are routinely enriched within network and information systems supporting your essential function(s). The enrichment of these alerts is performed in almost real time and as part of the original alert.

Alerts and the underlying detections are regularly reviewed and tested to ensure they are generated promptly and reliably, and it is possible to distinguish genuine security incidents from false alarms.

Alerts and the underlying detection rules are customisable and tuned to reduce false positives as well as optimising responses.

Detections and alerting may use off the shelf tooling and rules as well as custom tooling and / or rules.

You continuously monitor for user and system abnormalities indicative of adverse activity generating alerts based on the results of such monitoring.

Logs are monitored continuously in near real time.

C1.d Triage of Security Alerts

You contextualise alerts with knowledge of the threat and your systems, to identify those security incidents as well as responding to all alerts appropriately.

Not achievedPartially achievedAchieved
At least one of the following is true:All the following statements are true:All the following statements are true:

You do not triage alerts from your detection security technologies (e.g. AV, IDS).

You do not categorise alerts and incidents by type and priority / severity level.

You do not have Standard Operating Procedures (SOPs) / Playbooks / Runbooks available for use during triage.

You do not keep records of triage performed.

You do not have a sufficient understanding of normal user or system behaviour to make effective decisions within triage.

You investigate and triage alerts from some security tools and take action.

You have created, made available and use when appropriate, Standard Operating Procedures (SOPs) / Playbooks / Runbooks covering the most common use cases. These are regularly reviewed to ensure they remain effective.

You perform some triage and actions taken by monitoring and detection personnel are recorded.

You categorise alerts and incidents by type and priority / severity level.

Your understanding of normal user or system behaviour informs your decision making within triage.

You investigate and triage alerts from all security tools and take action.

You have created, made available and use when appropriate, Standard Operating Procedures (SOPs) / Playbooks / Runbooks covering all plausible use cases. These are regularly reviewed to ensure they remain effective.

You categorise alerts and incidents by type and priority / severity level.

You document all triage related activities performed by monitoring and detection personnel and these are used to drive improvements

Triage provides enough information for subsequent activities to be prioritised (e.g. the containment of damaging malware).

Your understanding of normal user and system behaviour, and threats, is sufficient for effective decision making within triage.

C1.e Personnel Skills for Monitoring Tools and Detection

Monitoring and detection personnel skills and roles, including those outsourced, reflect governance and reporting requirements, expected threats and the complexities of the network or system data they need to use. Monitoring and detection personnel have sufficient knowledge of network and information systems and the essential function(s) they need to protect.

Not achievedPartially achievedAchieved
At least one of the following is true:All the following statements are true:All the following statements are true:

There are no personnel who perform a monitoring and detection function.

Monitoring and detection personnel do not have the correct specialist skills.

Monitoring and detection personnel are not capable of reporting against governance requirements.

Monitoring and detection personnel have a lack of awareness of the essential function(s) the organisation provides, what assets relate to those functions and hence the importance of the log data and security events.

Monitoring and detection personnel have no awareness of other roles or tasks outside of security monitoring and detection that are relevant to the operation of your essential function(s).

Monitoring and detection personnel are overwhelmed with the amount of data and alerts they have to work with. Alert / triage fatigue is present.

Monitoring and detection personnel have some investigative skills and a basic understanding of the data they need to work with.

Monitoring and detection personnel can report to other parts of the organisation (e.g. security directors, resilience managers).

Monitoring and detection personnel are capable of following most of the required workflow(s).

Monitoring and detection personnel are aware of some of the network and information systems and your essential function(s), and can manage alerts relating to them.

Monitoring and detection personnel have some understanding of the operational context (e.g. people, processes, network and information systems that support your essential function(s)) to enhance the security monitoring function.

Monitoring and detection personnel deal with their workload and cases effectively.

You have monitoring and detection personnel who are responsible for the proactive and reactive analysis, investigation and reporting of monitoring alerts including both security and performance.

Monitoring and detection personnel have defined roles and skills that cover all parts of the monitoring and investigation process.

Monitoring and detection personnel follow policies, processes and procedures that address all governance reporting requirements, internal and external.

Monitoring and detection personnel are empowered to look beyond the fixed process to investigate and understand non-standard threats.

Monitoring and detection personnel are aware of the network and information systems and your essential function(s), related assets and can identify and prioritise alerts and investigations that relate to them. 

Monitoring and detection personnel drive and shape new log data collection and can make effective use of it.

Monitoring and detection personnel are capable of following all of the required workflow(s).

Monitoring and detection personnel have a sufficient understanding of the operational context (e.g. people, processes, network and information systems that support your essential function) to enhance the security monitoring function.

Monitoring and detection personnel deal with their workload and cases effectively as well as identifying areas for improvement.

C1.f Understanding User's and System's Behaviour, and Threat Intelligence (within Security Monitoring)

Threats to the operation of network and information systems, and corresponding user and system behaviour, are sufficiently understood. These are used to detect cyber security incidents.

Not achievedPartially achievedAchieved
At least one of the following is true:All the following statements are true:All the following statements are true:

Your organisation has no sources of threat intelligence.

You do not evaluate the usefulness of your threat intelligence or share feedback with providers or other users.

You have no awareness of the steps necessary to make best use of threat intelligence for security monitoring.

Threat intelligence is unreliable and / or is not actioned by the appropriate users or systems in a timely manner.

You have no established understanding of what abnormalities to look for that might signify adverse activities.

You do not receive updates for all your detection security technologies (e.g. AV, IDS).

You do not understand normal user and system behaviour sufficiently to be able to use abnormalities to detect adverse activity.

You know how effective your threat intelligence is (e.g. by tracking how threat intelligence helps you identify security incidents).

Your organisation may use threat intelligence services, but you do not necessarily choose sources or providers specifically because of your business needs, or specific threats in your sector (e.g. sector-based infoshare, software vendors, anti-virus providers, specialist threat intel firms, special interest groups).

The user and system abnormalities from past attacks and threat intelligence, on your 

and other network and information systems, are used to signify adverse activity.

You receive regular updates for all of your detection security technologies (e.g. AV, IDS).

You track the effectiveness of your threat intelligence and actively share feedback on the usefulness of Indicators of Compromise (IoCs) and other intelligence with the threat community (e.g. sector partners, threat intelligence providers, government agencies).

When using threat intelligence feeds, these have been selected using risk-based and threat-informed decisions based on your business needs and sector.

You make relevant, reliable and actionable threat intelligence available to the necessary users and systems promptly.

You contextualise threat intelligence and link it to the why and / or how attacks take place for security monitoring.

You understand normal user and system abnormalities fully, to such an extent that searching for system abnormalities is an effective way of detecting adverse activity (e.g. you fully understand which systems should and should not communicate and when).

The user and system abnormalities you monitor for are based on the nature of adverse activities likely to impact network and information systems supporting the operation of your essential function(s).

The user and system abnormalities indicative of adverse activity you use are regularly updated to reflect changes in network and information systems supporting your essential function(s) and current threat intelligence.

You possess the capability to share threat intelligence (e.g. ways to effectively detect adversaries) with the threat community / defender community (sector partners, threat intelligence providers, government agencies) when required.


Published

Reviewed

Version

4.0