Mongo DB is a NoSQL, scalable, open-source, high-performance, and document-oriented database. It is one of the best NoSQL databases in the market. Using caching with Mongo DB gives applications a significant performance boost by reducing trips to the database. However, caching Mongo DB data inside a distributed cache creates two copies of this data. One copy resides within Mongo DB, and one copy resides within the distributed cache. If an application changes the data inside Mongo DB directly, the data inside the cache becomes stale.
For high-transaction, multi-environment, and enterprise-level applications, this stale data is a big problem. In this blog, I’ll discuss how you can continue to use a distributed cache alongside Mongo DB and avoid the stale data problem at the same time.
Synchronize Cache with Mongo DB Using NCache
NCache is a well-known distributed cache for .NET Framework/ .NET Core, Java, and Node.js. NCache provides a vast set of features that can improve the performance of your application by leaps and bounds. Among these vast sets of features is a feature known as Notification Based Extensible Dependency. This feature addresses the stale data problem with MongoDB (which we discussed earlier).
Figure 1: Diagram depicting the working of NCache with Mongo DB
The NotifyExtensibleDependency class allows you to write your custom dependency logic in which you are notified of the changes taking place within the database through notifications. NCache processes these notifications by making use of a feature of Mongo DB (introduced in version 3.6) known as Change Stream. Once these notifications are processed, stale data within the cache is invalidated based on the code you have written.
The Change Stream logic is integrated inside the NotifyExtensibleDependency. You can modify this Change Stream to receive notifications on add, update, or delete operations that take place inside the database. But, the delete operations can’t be tracked. So, to track delete operations, an update operation with expiry is used.
It should also be kept in mind that Mongo DB doesn’t support Change Stream for Stand-Alone databases. You need to ensure that your Mongo DB database has a replica set so that the Change Stream feature works.
NCache Details NCache NotifyExtensibleDependency NCache Docs
Sync Cache with Mongo DB: A Quick Example
To use Mongo DB with NCache you need to configure Custom Dependency in NCache first and then deploy it in your client application afterward. This is explained below:
NCache Custom Dependency Configuration
To use the Notification Based Extensible Dependency feature of NCache with Mongo DB, you need to configure Custom Dependency on NCache first and deploy your respective DLL file(s) on the NCache Server.
In the GIF below, I have deployed a provider named MongoDbNotifyExtensibleDependencyProvider along with all of its dependencies on the cache. It implements the ICustomDependencyProvider interface. It is responsible for creating a dependency object for the CacheItem. For the dependency to be triggered properly, the NotifyExtensibleDependency class must be implemented. This class will listen to the MongoDB Change Stream and trigger dependencies upon data change in Mongo DB.
Also, in this example, I am using an open-source implementation of ICustomDependencyProvider and NotifyExtensibleDependency available on GitHub.
Figure 2: Notify Extensible Dependency Provider Deployment on NCache
NCache Details Custom Dependency Configuration GitHub Repository
Deployment on Client Application
For example, you wish to use Mongo DB as a database for your E-Commerce store and want to cache frequently used data inside NCache. Your store will contain data of over a million products, and millions of transactions will take place daily. NCache will help with reducing the load on your Mongo DB database, but the data inside the cache is prone to going stale.
Having stale data inside the cache can cause problems for you and your customers. Fortunately, the Notification Based Extensible Dependency feature of NCache mitigates the problem of stale data ensuring the optimal functionality of your E-Commerce store.
In the following example, a product is loaded from the database based on its ID. Then NotifyExtensibleDependency is created and registered against the fetched product. Finally, the product, in the form of CacheItem is inserted into the cache with the key specified.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Product product = LoadProductFromDatabase(productId); //This is the name of your provider deployed on a cache server string providerName = "MongoDbNotifyExtensibleDependencyProvider"; string key = "Product:" + product.Id ; IDictionary<string, string> parameters = new Dictionary<string, string>(); parameters.Add("ConString", "mongodb://**.**.**.**:27017"); parameters.Add("DatabaseName", "productscollection"); parameters.Add("CollectionName", "products"); CacheItem item = new CacheItem(product); item.Dependency = new CustomDependency(providerName, parameters); _cache.Insert(key, item); |
NCache Details NotifiyExtensibleDependency Cache Operations in NCache
Using Read-Through Provider
Another prominent feature of NCache is the Read-Through Provider. This feature allows you to look for data inside Mongo DB directly when it isn’t found in the cache. When the item is found inside the database, it is also stored inside the cache automatically; thus, saving you precious time. You can use this feature in your Mongo DB applications to further enhance your performance.
The following code example retrieves an item with read-through enabled, corresponding to the specified key “Product:1001” using the Get<> method.
|
1 2 3 4 5 6 7 8 9 |
// Specify the key of the item string key = "Product:1001"; // Specify the readThruOptions for read through operations var readThruOptions = new ReadThruOptions(); readThruOptions.Mode = ReadMode.ReadThru; // Retrieve the data of the corresponding item with reads thru enabled Product product = cache.Get(key, readThruOptions); |
NCache Details Read Through Caching Cache Operations in NCache
Conclusion
To sum up, NCache offers a flexible solution for synchronizing data between Mongo DB and the cache, thereby avoiding data inconsistency. NCache, an in-memory distributed caching solution, can be an ideal medium for Mongo DB to maintain up-to-date data for fast access and processing.
