Cache Data with MongoDB

MongoDB is a popular NoSQL database, particularly for modern .NET applications. However, it is still disk-based, which might lead to performance issues in situations with significant traffic, despite its flexibility, scalability, and powerful querying capabilities. This is where NCache, a distributed in-memory caching solution, comes in.

Introducing NCache as a caching layer for MongoDB allows .NET applications to drastically reduce database queries, enhance response times, and manage high transaction volumes efficiently.

Why MongoDB Needs Caching?

It is beneficial to .NET applications for the following reasons:

Flexible Document-Based Storage: This simplifies data organization due to its schemaless design.
Distributed Architecture: Enables replication and partitioning for scalability.
Indexes & Aggregation Framework: Makes complex queries efficient.
Change Streams: Captures real-time data modifications via Change Streams.

MongoDB Performance Bottlenecks

It has the following performance issues:

Read/Write Latency & Disk I/O Overhead: Since MongoDB is a disk-based database, frequent read and write operations lead to higher latency compared to in-memory solutions. Disk I/O overhead increases as data volume grows, impacting response times and overall system performance.
Large Dataset Queries: With increasing dataset size, query execution time increases because it takes longer to scan and process more documents. Indexes are useful in this regard but are not always enough, particularly when filtering or aggregating large volumes of data.
Scalability Challenges: Although it has built-in support for horizontal scaling using sharding, dealing with huge clusters may present replicational, load-balancing, and networking complexities. Performance optimization becomes critical as traffic volume rises.
High Latency for Frequently Accessed Data: Queries for frequently accessed data can lead to performance degradation if they repeatedly hit the database instead of being served from a faster storage layer. This causes a boost in resource utilization and delays in response times.

Figure: Distributed Cache using MongoDB

Why Use NCache as a Distributed Cache for MongoDB?

NCache, an in-memory distributed cache, is the best option for your MongoDB application:

In-Memory Performance: NCache is a 100% in-memory caching solution, eliminating disk I/O and ensuring ultra-low latency for faster data retrieval.
Client Cache (InProc Speed): Resides on the application server inside the application process, minimizing network calls and ensuring rapid access.
Linear Scalability: Dynamically scales out by adding cache servers as data load grows, preventing bottlenecks and ensuring high performance.
High Availability: Ensures uninterrupted access with automatic replication and failover, keeping the cache operational even during server failures.

Architecture: How NCache Fits into Your MongoDB Application

NCache seamlessly fits into your application due to its dynamic architecture:

Supports Read-Through and Write-Through caching to reduce direct database queries.
Integrates seamlessly with .NET 8 and cloud environments (Azure, AWS, Kubernetes).

Implementing MongoDB Caching with NCache

You can easily implement MongoDB caching with NCache as demonstrated below:

Caching Query Results in MongoDB

ICache cache = CacheManager.GetCache("MongoDBCache");

string cacheKey = "Customer:EINSTEIN";

Customer customer = cache.Get<Customer>(cacheKey);

if (customer == null)

{

var collection = new MongoClient("mongodb://your-mongodb-url")

.GetDatabase("ECommerceDB")

.GetCollection<Customer>("Customers");

customer = await collection.Find(x => x.Id == "EINSTEIN").FirstOrDefaultAsync();

if (customer != null)

{

cache.Insert(cacheKey, new CacheItem(customer)

{

Expiration = new Expiration(ExpirationType.Absolute, TimeSpan.FromMinutes(5))

});

}

Caching MongoDB Collections

You can easily cache MongoDB collections as below:

Cache Collection as a Single Item

Storing an entire collection as a single cache item allows for quick retrieval in one operation, reducing cache lookups and improving batch processing or UI rendering performance.

var customersInGermany = await collection.Find(x => x.Country == "Germany").ToListAsync();

if (customersInGermany.Count > 0)

{

ICache cache = CacheManager.GetCache("MongoDBCache");

var cacheItem = new CacheItem(customersInGermany)

{

Expiration = new Expiration(ExpirationType.Absolute, TimeSpan.FromMinutes(10))

};

cache.Insert("Customers:Germany", cacheItem);

}

Cache Collection Items Separately

Storing each item separately ensures individual access while using tags to group related items, making bulk retrieval faster and improving query performance.

foreach (var customer in customersInGermany)

{

var cacheItem = new CacheItem(customer)

{

Tags = new Tag[] { new Tag("Customer:Country:Germany") },

Expiration = new Expiration(ExpirationType.Absolute, TimeSpan.FromMinutes(10))

};

cache.Insert($"Customer:CustomerID:{customer.Id}", cacheItem);

}

Synchronizing Cache with MongoDB Using Change Streams

To ensure data consistency between MongoDB and the cache, change streams allow for real-time synchronization by detecting inserts, updates, and deletes. Whenever data changes, NCache can remove or update the corresponding cache entry, preventing stale data issues.

var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<Customer>>()

.Match("{ operationType: { $in: ['insert', 'update', 'replace', 'delete'] } }");

var cursor = collection.Watch(pipeline);

await cursor.ForEachAsync(change =>

{

string cacheKey = $"Customer:CustomerID:{change.FullDocument.Id}";

cache.Remove(cacheKey);

});

Querying Cached Data in NCache

NCache allows users to query cached data using SQL queries, reducing database load while improving application performance. Indexed queries enable faster lookups and filtering on cached objects.

// Connect to NCache

ICache cache = CacheManager.GetCache("MongoDBCache");

// Define NCache search query

string query = "SELECT * FROM Models.Customer WHERE Country = ?";

var queryCommand = new QueryCommand(query);

queryCommand.Parameters.Add("Country", "Germany");

// Execute query on cache

ICacheReader reader = cache.SearchService.ExecuteReader(queryCommand);

while (reader.Read())

{

string contactName = reader.GetValue<string>("ContactName");

Console.WriteLine($"Contact Name: {contactName}");

}

How to Configure NCache for MongoDB?

To configure NCache for MongoDB, follow the steps below:

A. Install NCache Client

To install the NCache client in a .NET application, use NuGet to add the required package:

1	Install-Package Alachisoft.NCache.SDK

Once installed, configure your application to use NCache for caching data.

B. Install NCache Server

To install NCache Server on Linux:

docker pull alachisoft/ncache:latest

docker run -d --name ncache-server --network host alachisoft/ncache:latest

docker exec -it ncache-server ncache service start

C. Configure NCache for MongoDB

To integrate MongoDB with NCache, follow these steps:

Create and Configure Cache:

1 2	New-Cache -Name MongoDBCache -Size 1024 -Topology PartitionReplica -Server 192.168.1.2 Add-Node -CacheName MongoDBCache -Server 192.168.1.3

Modify .NET Application to Use NCache:

ICache cache = CacheManager.GetCache("MongoDBCache");

var collection = new MongoClient("mongodb://your-mongodb-url")

.GetDatabase("ECommerceDB")

.GetCollection<Customer>("Customers");

var customer = await collection.Find(x => x.Id == "EINSTEIN").FirstOrDefaultAsync();

if (customer != null)

{

cache.Insert("Customer:EINSTEIN", new CacheItem(customer)

{

Expiration = new Expiration(ExpirationType.Absolute, TimeSpan.FromMinutes(10))

});

}

Conclusion

Using NCache with MongoDB enhances application performance by reducing database load, improving response times, and ensuring real-time data consistency. By leveraging in-memory caching, applications benefit from faster data access, lower latency, and seamless scalability. NCache’s distributed architecture provides high availability and ensures fault tolerance, making it an ideal choice for modern, high-traffic .NET applications utilizing MongoDB.