Client Cache

NCache cluster topologies are meant to provide the best performance and scalability for your applications. With growing business needs, applications need to process more client requests and data. Adding more nodes to the distributed cache seamlessly provides linear scalability. Like a hardware processor cache, the client cache takes the performance of your applications to a greater level by bringing the hot data set closer to your application, even inside the application's process with the InProc mode.

Consider the example of an E-Commerce application. The application frequently accesses the product catalog and data of currently active users. Such data can be kept in the client cache running on the client box. On one side, keeping this data in the client cache boosts the application's performance by avoiding trips to the database and clustered cache. On the other hand, it offloads many Read/Write operations from the cluster cache by enabling the cluster cache to take more requests. And, this performance gain is without compromising on data consistency. The client cache keeps its data synchronized with the cluster cache. How the client cache synchronizes with the cluster cache is explained in the following sections.

Plug & Play: Using the client cache is quite simple. No code changes are required at the application end. It is a simple configuration option. You can create a client cache through the NCache Management Center or the NCache supported Powershell cmdlets. Once the client has been configured, the client applications will automatically start using it. For already running applications, an application restart is required.

All CRUD operations that take cache keys as input, such as Add, Get, Insert, and Remove, are routed through the client cache. Read operations first look for the data inside the client cache. The Client cache returns the data (if found). Otherwise, the read operation is executed against the cluster cache. Data returned from the cluster cache is returned to the application after it has been added to the client cache. So, the next Read call for the same data will be served from the client cache. For Bulk Read operations, only the data missing in the client cache is fetched from the cluster cache.

Key-based Write operations like, Add and Insert, are first performed on the cluster cache. Upon their successful completion, data is added to the client cache and then forwarded to the application. Other client cache instances are updated through a background data synchronization mechanism, which is explained later.

The client cache holds only a subset of cluster cache data. Therefore, all the other non-key-based operations like GetGroup, SQL queries, and GetByTags, etc, are directly performed on the cluster cache.

Client Cache: Isolation Modes

Client cache runs on the client node where your applications are running. Depending on your performance needs and the application architecture, you can choose one of the following process-level isolation modes supported by the client cache.

In-Proc

As the name suggests, the client cache runs inside the application process, eliminating inter-process communication. User data is kept in object form to avoid the deserialization cost. This mode provides maximum performance to the application. As the client cache is hosted inside the application process, the data inside the client cache is not shared among other application instances. Each instance of the application hosts a dedicated client cache instance. Although InProc mode provides maximum process, it is suitable only if:

The application hot data set is not too large.
The application only has a few instances on each client node with enough physical memory. Remember that each client cache instance holds its copy of data.
The application life cycle is long enough to reap the benefits of the client cache. Remember that the client cache life cycle depends on the life cycle of the application. When the application goes down, all data inside the client cache is lost as well. Applications with short life cycles would shut down or restart before the client cache is entirely populated.

Each application has its own hot data set, which is different from other applications.

OutProc

This mode provides process-level isolation for the client cache. The client cache runs in its dedicated process on the client node. Applications communicate with the client cache over TCP sockets. Multiple application instances can communicate with the same client cache, hence, sharing data. Although InProc outperforms OutProc mode in performance, OutProc mode comes with its own set of advantages.

Multiple applications running on the same client machine communicate with the same client cache. Multiple applications share data. Data loaded or updated by one application becomes available for other applications.
Application restart does not result in client cache data loss.
Less physical resources like RAM and CPU are required to run the client cache in OutProc mode as compared to InProc mode when each application process holds its copy of the client cache.
Synchronization of client cache data (explained later) with cluster cache puts less burden on the cluster cache since you run a single client cache instance per client machine.

Note

If the OutProc client cache is down, the application will directly perform operations on the cluster cache. When the client cache starts again, it will automatically connect with the client cache. You can change this behavior by setting the skip-client-cache-if-unavailable flag in client.ncconf. If the flag is set to false, cache operations will fail if the client cache is down.

Synchronizing Data With Cluster Cache

Despite an easy Plug & Play setup, we can not ignore the fact that the client cache holds a copy of cluster cache data. Changes made to the data in a cluster cache should be propagated to the client cache. Multiple client caches running with either InProc or OutProc mode may exist for a given cluster cache. Data changes made by the client application are performed on the client cache instance to which the application is connected. Therefore, this instance of the client cache is automatically synchronized. However, other instances of the client cache are unaware of these changes. The changes made to cluster cache data are synchronized with these client cache instances through a background data synchronization mechanism explained below:

When data is added to the client cache, it registers data change notification with the cluster cache for the given data.
The cluster cache keeps track of each CacheItem that a client cache holds and monitors the changes made to the data.
When data is updated/removed from the cluster cache, the cluster cache records these changes.
A dedicated background worker thread inspects the data changes every second and determines which client caches should be notified of the changes. It sends a notification to the affected client caches that data has been changed.
Another dedicated background worker thread running in the client cache is responsible for synchronizing data changes with the cluster cache upon receiving the data change notification. As soon it receives the notification, it requests the cluster cache and asks for data updates. We call this synchronization mechanism Client cache polling.
This worker thread running in the client cache polls for the data changes every 10 seconds if it has not received any notification from the cluster cache to handle the cases where it may have missed a notification due to connectivity loss between the client cache and cluster cache.

This powerful synchronization mechanism makes sure that the client applications are always served with the latest data from the client cache with added performance and scalability.

Synchronization Modes

Along with the background data synchronization mechanism, the client cache supports the following two synchronization modes.

Optimistic Synchronization

This is the default synchronization mode of the client cache. When an application fetches data from the client cache and the client cache holds that data, data is simply returned to the application. Synchronization is performed in the background, as explained above.

Pessimistic Synchronization

The background synchronization mechanism is suitable for most of the applications and provides the optimum performance and scalability to the application. However, for applications that are more sensitive and want to make sure that they are always served with the latest data, the pessimistic mode is designed for them. With this mode, when an application fetches data from the client cache, and the client cache holds that data, the client cache verifies the item version with the cluster cache. If an updated version of data is found in the cluster cache, it is fetched and updated into the client cache. Thus application is guaranteed to get the latest version of the CacheItem.