How to Sync Cache with Cosmos DB?

Azure Cosmos DB is the new NoSQL cloud-based database solution that has gained a lot of popularity in recent times. Everyone knows that using caching with Cosmos DB boosts application performance and optimizes transaction costs by reducing trips to the database.

However, caching Cosmos DB data in a distributed cache can lead to the creation of two copies of this data, one in the database and one in the cache. If any application that does not have access to your cache directly changes this data in Cosmos DB, your cache is not aware of it. This results in your cache having an older copy of the data as compared to the database.

This stale data is a major problem in high-transaction, multi-environment applications where working with outdated data leaves a big mark on the performance. In this blog, I’ll discuss how you can continue using caching and still avoid this “stale data” problem.

Cache Synchronization with Cosmos DB Using NCache

NCache is a very powerful distributed caching solution with an abundant set of powerful features. Just like any distributed cache, NCache sits between the database and the application itself and caches the data from the database to be used by the application; thus, reducing the read/write time exponentially. It also solves the problem of stale data (discussed earlier) by a feature known as NotifyExtensibleDependency.

Figure 1: A diagram illustrating the working of NCache with Cosmos DB

Through NotifyExtensibleDependency, you can write your own custom data dependency logic in which a notification from a database (Cosmos DB in this case) is processed by NCache by making use of the Change Feed mechanism of Cosmos DB.

Any modification in the Cosmos DB can be captured using the Cosmos DB Change Feed processor logic that is integrated into the NotifyExtensibleDependency code. This is how the event handling mechanism can be used to enforce cache invalidation in response to modifications in the database content, thus ensuring that stale data does not persist in the cache.

You also have the option of using the ReadThru provider feature of NCache to further improve the performance of your Cosmos DB application. Through this feature, you can enable the cache to directly look into the database for items that don’t exist in the cache. Not only does the cache look for the requested items in the database (Cosmos DB), but also stores the requested items inside the cache upon successful discovery.

NCache Details NotifiyExtensibleDependency Read Through Caching

Sync Cache with Cosmos DB: A Quick Example

For example, you are using Cosmos DB as a database for your E-Commerce store and cache frequently used data in NCache. Your store contains tens of thousands of products and on average a million transactions take place daily. NCache helps you with reducing the load on your Cosmos Database, but there is a chance that the data in the cache is going stale. As discussed above, NCache solves this problem by its NotifyExtensibleDependency feature that ensures optimal functionality of your E-Commerce store.

The following code example demonstrates the usage of the NotifyExtensibleDependency feature. In this example, a product is loaded from the database based on its ID. Then NotifyExtensibleDependency is created and registered against the fetched product. Finally, the product, in the form of CacheItem is inserted into the cache with the key specified.

Also, the CosmosDbNotificationDependency is the name of the provider which has been deployed on your cache. This provider implements the ICustomDependencyProvider interface.

Product product = LoadProductFromDatabase(productId);

string providerName = "CosmosDbNotificationDependency"; //This is the name of your provider deployed on cache server

string key = "Product#" + product.Id ;

IDictionary<string, string> parameters = new Dictionary<string, string>();

parameters.Add("Key", key);

parameters.Add("CacheId", "myCache");

parameters.Add("EndPoint", "");

parameters.Add("AuthKey", "");

parameters.Add("DatabaseName", "demoDatabase");

parameters.Add("MonitoredCollection", "Customers");

parameters.Add("LeaseEndPoint", "");

parameters.Add("LeaseAuthKey", "");

parameters.Add("LeaseDatabaseName", "demoDatabase");

parameters.Add("LeaseCollection", "leases");

CacheItem item = new CacheItem(product);

item.Dependency = new CustomDependency(providerName, parameters);

_cache.Insert(key, item);

For further details related to the implementation, you can check out our GitHub repository.

NCache Details NotifiyExtensibleDependency Cache Operations in NCache

Read-Through Provider for Cache

The Read-Through feature of NCache (as the name suggests) enables you to read through the cache directly into your database (Cosmos DB) when items are not found in the cache. This feature basically saves you the hassle of looking in the database for items yourself when those requested items are not found in the cache. Through this feature, not only does NCache read your database for the requested items, but also inserts them in the cache for you for quick retrievals in the future.

The following code example retrieves an item with read-through enabled, corresponding to the specified key “Product:1001” using the Get<> method.

// Specify the key of the item

string key = "Product:1001";

// Specify the readThruOptions for read through operations

var readThruOptions = new ReadThruOptions();

readThruOptions.Mode = ReadMode.ReadThru;

// Retrieve the data of the corresponding item with reads thru enabled

Product data = cache.Get(key, readThruOptions);

NCache Details Read Through Caching Cache Operations in NCache

Conclusion

To sum up, NCache offers a flexible solution for synchronizing data between Cosmos DB and the cache, thereby avoiding data inconsistency. NCache, an in-memory distributed caching solution, can be an ideal medium for Cosmos DB to maintain up-to-date data for easier access and processing.

NCache Details Download NCache Cloud Deployment Options for NCache.