Transaction Monitoring for online services

Introduction
Transaction Monitoring (TxM) is an essential function that can help prevent both monetary and reputational damage. This guidance will outline the use and relevance of TxM, the attacks it can mitigate, and the techniques used to monitor and detect suspicious transactions.
This guidance is aimed at service owners and security specialists involved in the provision of online services.
Note:
The aim of TxM is to help detect attacks. It should be applied as part of a suite of security measures to protect online systems/services, and should not be relied upon alone. That is, TxM should complement your overall approach to security monitoring, helping to provide an end-to-end view of online transactions, and the attacks that may affect them.
What is Transaction Monitoring?
TxM is a capability that enables the owners and operators of online services to detect and respond to abnormal and suspicious user activity and behaviour. However, in order to detect this anomalous behaviour it is important to understand what ‘normal’ looks like. This can be achieved using techniques such as customer profiles to get a view of what usual customer behaviour looks like, these along with further techniques will be outlined below.
The following diagram shows a high level outline of how a TxM system works:

Figure 1: High-level overview of transactional monitoring system
Who should use TxM?
All forms of security monitoring, including TxM, add a layer of trust to a system. TxM is not only of interest to banks or financial companies; any online service which provides the functionality for an exchange of something of value should be monitoring transactions.
TxM systems usually need to be tailored to each situation, so ‘out of the box’ (or poorly implemented) solutions can cause problems, such as excessive false positive alerts that would then require a large amount of manual work. However, a well-implemented TxM system can provide huge benefits to any organisation. For example, they can help detect attacks that are often missed, such as those that exploit vulnerabilities in:
- the logic of a system service or process
- a customer’s endpoint (for example Man-in-The-Browser [MiTB] attacks)
Impact of malicious transactions
Malicious transactions can have a large impact on organisations, and can can be grouped into the following losses:
- financial loss; either small or large-scale; it can be devastating for parties involved (including the possibility of bankruptcy)
- reputational loss; loss of reputation can have a huge business impact because customers will no longer have trust
- data compromise; an online transaction may involve the exchange of sensitive information and data between transacting parties
Some TxM solutions categorise or score transactions in terms of risk to inform the amount and kind of analysis that is conducted. When determining the risk associated with an online transaction it is important not only to consider factors relating to the financial value of a transaction. For example, transactions made from unusual geographic locations or at unusual times may also be deemed higher risk.
Advanced TxM solutions can be costly to implement and manage. You should consider the impacts malicious transactions could have, and only implement those TxM capabilities that are proportionate to the transaction-related risks you face.
Risks, threats, and attacks
The type of TxM system you implement must be based on a clear understanding of the cyber security risk the online transactions pose to your organisation, its business, and your online systems and services. There are many factors affecting the attractiveness of your online service to cyber criminals and other threats, and the overall transaction risk, including:
- the potential for fraud and other malicious attacks against your service
- the volume and frequency of online transactions
- the nature of online transactions (such as how and where they are conducted from)
- the impact resulting from fraudulent or malicious transactions
There are a huge range of attacks that can be carried out to compromise a transaction. Some of the most common attacks include credential theft, man-in-the-browser attacks, phishing emails and session hijacking. These need to be considered alongside other malicious attacks that may affect your online system or service such as denial of service (DoS) attacks.
These types of attacks aim to look like a legitimate interaction between a customer and your service, making it hard to tell a fraudulent transaction from a legitimate one. You should understand that whilst TxM will help you detect the transactions enabled by these attacks, you will still need to make sure that you have applied security controls to protect against them, like those described in the NCSC’s secure design principles.
The types of threats aiming to exploit online transactions will have different motivations. Threats may include:
- cyber criminals, who regularly abuse online services to commit fraud
- nation state actors, who are generally interested in accessing really sensitive or valuable information that may give them a strategic or political advantage (or with the aim to disrupt/interfere with the way online services work)
- terrorists, looking to fund their activities or spread propaganda and disruption activities, they generally have less technical capabilities
- malicious hackers, individuals with varying degrees of expertise often acting in an un-targeted way, perhaps to test their own skills or cause disruption for the sake of it
A TxM system may also help detect automated attacks against your online system from bots. A bot is a small piece of software that often is used to perform automated, repetitive, pre-defined tasks. Once a bot gains access to a system, it’s unlikely to be detected by normal security controls. To avoid compromise due to bots, your TxM system should include capabilities to detect the behavioural signs of an automated attack.
Detecting suspicious alerts
A TxM system should alert when anomalous or suspicious transactions are detected. While it is difficult to spot fraudulent transactions from legitimate ones, understanding what suspicious looks like will make this process much easier. Suspicious transactions may involve:
- a sequence of user activity that does not match users' normal behaviour
- rapid user interactions (indicative of a bot or automated attack)
- a transaction not following agreed policies in place (such as a larger transaction request without prior agreement/permission)
Clearly, if you don’t understand your customers' ‘normal’ activity, then you won’t be able to spot suspicious or atypical behaviours. ‘Knowing your customer’ is an important first step, and is particularly relevant with customers that carry out numerous high-risk transactions, as the threat against them is more significant. Further, ensuring TxM is part of the wider logging and monitoring system is important to gain a full view of transactions and behaviours within the service, as simply relying on common themes will not always spot anomalous transactions.
Techniques for monitoring transactions
Once transaction data is captured, it must be analysed to detect anomalous activity so that the appropriate response is triggered. Although there are ‘tell-tale’ behaviours that could indicate malicious transactions, using simple rule-based models can result in masses of false positive alerts. As discussed using an ‘out of the box’ rule-based solution will probably identify fraudulent transactions, but these will be outnumbered by false positives, making it harder to determine the actual malicious transactions. A more tailored approach (for example one that includes detailed customer and device profiles) whilst taking longer to set up, will lead to more effective TxM.
Bespoke techniques

Figure 2: Use of Customer Profiles
Some bespoke techniques that can be implemented include:
- Customer profiles: these are profiles kept on each customer, outlining information given upon sign-up such as email, postal address, phone number as well as behaviour patterns/activities such as number of transactions, size of transactions, and the location of the user. These must be kept up to date.
- Device profiles: these profiles can be part of the customer profile. They provide information on the devices that are commonly used to perform transactions, enabling alerts to be raised where these profiles change. Changes might include device, operating system or browser type, geo-location of the IP address (different country or internet service provider).
- Analysis of historical transaction: examining past transactions can be useful for identifying patterns that may indicate long-term or even automated fraud.
Techniques like this can also be augmented using technologies such as machine learning. However, this will require a large amount of resource and time. Machine learning modules need to be trained with suitable data for them to be able to detect fraudulent transactions effectively, and any change in data used may render a once reliable system almost useless. Despite these challenges, if there is enough resource to build a reliable machine learning module for the TxM system, they can prove to be highly effective.
Procedures and responses
To ensure efficiency of the TxM system, there is an expectation to have some form of policy in place, so all parties involved are following the same processes. This ensures minimal human error as the actions taken should all follow defined procedures.

Figure 3: Procedures and Responses
One procedure that has to be defined is what to do when a suspicious transaction has been detected. There are a few responses that can be considered:
- Response 1: Allow the transaction to go through. This would be for a situation where it is very clear that the user is carrying out a genuine transaction, for example where similar transactions have been carried out in the past using the same device profiles.
- Response 2: Immediately decline the transaction and/or block access. This would only be when a high-level of confidence has been achieved that the transaction or user is fraudulent.
- Response 3: Further investigation is required. This step can be carried out in a number of different ways including additional internal checks on the transaction before it is verified, or an extra step of authentication is put in place to strengthen the confidence that the user is who they claim to be.
Listed above are some example procedures, where you have the ability to intervene in real time. However, this isn’t always possible. The likelihood of catching every fraudulent transaction in the act is very low, and therefore procedures also need to define responses to fraudulent transactions that have taken place undetected. For example, a transaction at the time that looked ‘normal’, but after a series of other transactions appears suspicious, could be fraudulent. In this case, responses might include reporting to either an Incident Management team/SOC internally, or depending on the severity/size of the transaction, reporting externally to agencies such as local law enforcement, the National Crime Agency, and if personally identifiable information has been compromised, the ICO.
Managing your TxM system
Managing your TxM system ensures it is functional and effective. The four main steps that should be included in the management of your system are:
- ensuring systems are kept up to date and patched
- routine security testing of the TxM system
- updating customer/device profiles
- checking the effectiveness of any rulesets being used
Regularly updating your system is key to preventing vulnerabilities being exploited as well as improving the overall security and functionality of your system. Accurate customer and device profiles maintain the effectiveness and efficiency of your transaction monitoring system. These need to be updated and validated inline with changes to this data. Keeping profiles up to date will also help to minimise false positives that could stem from out-of-date customer or device information. Finally, it is important to make sure that the management of your TxM system is part of any policy or framework you use to govern and direct TxM within your organisation.


