Let’s say you pay at a mart using your card. Now you are at home, haven’t gone out in a while and you receive a message from your bank stating you have debited $20000. How? What you purchased cost you a mere $120! This will entail a long process of verifications with your bank, and the bank may take months in tracing and resolving the fraudulent transaction.
It would be much easier if a fraud detection mechanism is incorporated in the system so that it determines invalid transactions and fails them in real-time. However, this decision has to be instant and the system should train itself continually based on the user’s past behaviors. Hence, you can’t expect customers to wait at the counter while your fraud detection system takes time in detecting any anomaly. You need to have an efficient as well as faster system. This is possible if you store the data in-memory using a cache such as NCache and process it within milliseconds.
Figure 1: Fraud Detection using NCache
Generally, a fraud detection system contains three subsystems, as illustrated in Figure 1:
- Near Real-Time (NRT) System: This system processes the input and the results are generated using pre-defined patterns and rules, instead of performing the actual calculation. Moreover, the results must be in milliseconds as it responds directly to the user. The main purpose of this system is high throughput with low latency which means this system should ideally be in local memory to achieve higher performance.
- Input-Processing System: This system can take up to minutes to hours for processing the input and adjusting the parameters of the NRT system. This updated information is fed to the offline processing system, that further processes machine learning algorithms on it.
- Offline-Processing System This system can take from hours to months, to respond. This system improves the model, trains the data, and feeds this upgraded data back to the NRT system for better results.
NCache as In-memory Cache and Pub/Sub Messaging Bus
NCache is an in-memory distributed cache that offers powerful Pub/Sub messaging and other features such as distributed data structures. Keeping in view the systems involved in a fraud detection system, NCache seamlessly fits in the Near-Real-Time System as it requires blazing fast results. This is possible as NCache can be used for communication between multiple elements as well as storing data nearest to your application, further reducing network calls to fetch data.
We have created a sample fraud detection system in .NET that includes NCache to improve performance and throughput. You can find the working sample here at GitHub.
At an upper level, the application works as such:
- The client application communicates with the Transaction Request Manager that manages the transaction of the given customer.
- The transaction information is published as a message to the Pub/Sub topic.
- The Fraud Detection Manager subscribes to this topic and receives the transaction message.
- The fraud detection logic is performed on the item and the result is sent to the transaction manager that is sent back to the client application.
- The customer information is then stored in the cache using Distributed data structures such as Lists. This will help in making decisions on future transactions much easier.
- The data is persisted in the database using Write-behind.
Managing Transactions with Pub/Sub
The Transaction Request Manager is initialized with the cache. Transaction Request Manager creates the transaction as a message and publishes it to its topic.
|
1 2 3 4 5 |
public void CreateRelevantSubscription() { MessageReceivedCallback transactionmessageReceivedCallback = new TransactionCompletedMessage(this).MessageReceivedCallback; transactionSubscription = base.CreateRelevantSubscriptions(Topics.REPLIESTOPICS, transactionmessageReceivedCallback); } |
Whenever a transaction is made through the Transaction Request Manager, it first creates a message from the customer information and then publishes this as a message on the transaction topic. This topic is subscribed by the Fraud Detection Manager, which receives the message and performs fraud detection logic on the incoming message and sends the results back to the Transaction Request Manager.
|
1 2 3 4 5 |
// Sending messages on the respective topic if (customer != null) { ncache.PublishMessageOnTopic(Topics.TRANSACTIONTOPICS, CreateTransactionFromCustomer(customer), null); } |
Detecting Fraud in Real-Time
The Fraud Detector contains complete logic for verifying if a transaction is valid or not. It behaves like a handler for performing all transactions and establishing a connection with the cache storage. The Fraud Detection Manager is initialized by creating a topic in NCache and subscribing to this topic. The callback for the Fraud Detection Manager topic exists in the Transaction Request Manager logic as it sends the results of the fraud detection logic back to the Transaction Request Manager.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
public void InitiliazeFraudManager() { base.CreateRelaventTopics(Topics.REPLIESTOPICS); base.CreateRelaventTopics(Topics.TRANSACTIONTOPICS); CreateRelevantSubscription(); } public void CreateRelevantSubscription() { // Initializes the relevant subscribers with their callbacks MessageReceivedCallback transacionManagermessageReceivedCallback = new StartTransactionMessage(this).MessageReceivedCallback; transactionSubscription = base.CreateRelevantSubscriptions(Topics.TRANSACTIONTOPICS, transacionManagermessageReceivedCallback); } |
In Fraud detection logic, first, a customer data is fetched from memory, if it is present there, we verify if any last in-memory transaction was fraudulent, if yes, then we declare this transaction as a failure too. If no such information is received, and all in-memory transactions are valid then we perform the already learned logic on this data.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
if (customerInfo != null) { if (learntLogic.FraudFoundInLastTransactions(transactionMessage, customerInfo)) { result = Result.Faliure; } } if (result != Result.Faliure) { bool isValid = learntLogic.IsTransactionValid(transactionMessage); if (!isValid) result = Result.Suspicious; } |
In this learned logic, we use multiple factors to determine the validity of the transaction. We check if a transaction has originated from a valid IPAddress and a valid email address that lies in a proper domain. Moreover, the email shouldn’t exist in a range of suspicious emails. It is also checked if the transaction originated from any suspicious location. If any of the above criteria fails, the failure weight is incremented. If the failure weight is greater than the decision weight, the transaction as invalid. You can find the code for this logic at GitHub.
Caching Results and Persist them to Database
Once the result has been declared, the result is added/updated in the cache.
|
1 2 |
FraudRequest fraudRequest = CreateFraudRequest(transactionMessage, result); ncache.UpdateCustomerInfoInCache(customerkey, fraudRequest); |
The cache is configured for the data source. For storing customer information in the in-memory cluster, we have used NCache’s distributed lists, which are one of the distributed data structures. The information of the customers is also persisted to the database using Write-Behind, which asynchronously updates the customer information.
|
1 2 3 4 5 6 7 8 9 10 11 |
IDistributedList list = cache.DataTypeManager.GetList(key); if (list == null) AddCustomerInCache(key, null, cutomerInfo); else { // Update info of a customer against its id list.WriteThruOptions = new WriteThruOptions(WriteMode.WriteBehind); list.Add(cutomerInfo); } |
Once cached, a new message is created from this result and published back on the topic subscribed by the Transaction Request Manager. It receives the message and publishes the result back to the client application.
Conclusion
NCache is a distributed in-memory .NET cache that comes with multiple features like Pub/Sub, distributed data structures, data source providers and more. The purpose of storing and processing data in local memory is eliminating the cost of network transfer, as all processing is done through the cache, and results are later updated on the database.
Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck with the upcoming articles.