The advent of distributed caching systems has easily addressed the challenges of database overload and performance lag. Today, these distributed caches can efficiently manage millions of data transactions per second. However, a significant issue remains; many enterprise-level applications need to track changes within vast amounts of data. These applications have to make critical business decisions based upon these changes given the considerable size of this data (thousands of Gigabytes), monitoring specific changes is quite a challenge.
As such, NCache, an in-memory distributed cache, offers a feature known as Continuous Query. This blog explains how CQ works, how you can configure it in NCache, and the benefits it provides for managing and monitoring data changes effectively.
Continuous Query to Monitor Changes in NCache
NCache’s Continuous Query feature allows you to keep a track of changes taking place in a selective dataset within the distributed cache cluster. This dataset is defined through the SQL-like OQL (Object Query Language) queries. The changes that take place within this dataset are propagated to the applications (that have registered callbacks for events) in the form of Cache Level Events.
Regardless of the size of your cache cluster, you will only be notified of changes within the dataset you defined and registered in the cache. This is how CQ achieves application decoupling, i.e., by filtering data through OQL queries, ensuring that applications don’t overlap with each other.
It should also be noted that Continuous Query itself doesn’t change your application data. Instead, it provides a mechanism for monitoring and sharing data among applications at runtime through events. Developers can then register callbacks for Continuous Query events and define their business logic to determine how applications handle this data.
Figure: A diagram depicting the working of Continuous Query
Configuring Continuous Query in NCache
Configuring CQ in NCache is a two-step process. Once completed, your application will receive notifications against the defined dataset and respond accordingly. These steps are as follows:
Step 1: Register Query and Notifications
Begin by creating a Continuous Query that specifies the criteria for the dataset you’re interested in. The events will be triggered based on this Continuous Query. When registering for CQ notifications, you must subscribe to at least one event type (add, update, or remove). After defining the query, register the relevant callbacks with it and then register the query against the cache server using the RegisterCQ method.
The code example below depicts how all of this happens.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
// Query for required operation string query = "SELECT $VALUE$ FROM Alachisoft.NCache.Samples.Data.Product WHERE Category = ?"; // Create query command and add parameters var queryCommand = new QueryCommand(query); queryCommand.Parameters.Add("Category", "Beverages"); // Create Continuous Query var cQuery = new ContinuousQuery(queryCommand); // Item add notification // EventDataFilter.DataWithMetadata returns the keys with the data and meta added with data cQuery.RegisterNotification(OnChangeInQueryResultSet, EventType.ItemAdded, EventDataFilter.DataWithMetadata); // Register continuousQuery on server cache.MessagingService.RegisterCQ(cQuery); |
Step 2: Defining Callback for Events
You need to define a callback for cache level events: ItemAdded, ItemUpdated, or ItemRemoved based on your requirement. Through these events, you must define your business logic to dictate what your application should do if one of these events is fired. In the following example, a callback is registered for the event ItemAdded.
|
1 2 3 4 5 6 7 8 9 |
public void OnChangeInQueryResultSet(string key, CQEventArg arg) { switch (arg.EventType) { case EventType.ItemAdded: Console.WriteLine ($"Item with key '{key}' has been added to result set of continuous query"); break; } } |
Step 3: Unregister Notifications from Continuous Query
NCache gives you the option to unregister notifications from the Continuous Query when you no longer require them. This is done using the UnRegisterNotification method. You can use this method to unregister notifications for add, update, and remove operations, altogether or a specific event. If you’re interested in receiving notifications for specific types of events (e.g., add and remove) but not others (e.g., update), you can unregister the update notifications while still keeping the add and remove notifications registered. It is important to consider that when a client unregisters a notification from a Continuous Query, it affects only that client. Other clients with the same notifications remain unaffected, allowing independent subscription management.
In the following example, an event notification for an item added notification is unregistered from a CQ. However, keep in mind that you cannot unregister all CQ notifications through this method. For that, a separate method UnRegisterCQ is used.
|
1 |
cQuery.UnRegisterNotification(OnChangeInQueryResultSet, EventType.ItemAdded); |
Step 4: Unregister Continuous Query from Server
NCache also gives you the option to unregister the entire CQ from the cache cluster using the UnRegisterCQ method. When a client uses the UnRegisterCQ method, the result set isn’t immediately removed from the server. If it is the last client, the result set will be removed from the cache. If not, the result set remains in the cache, but that specific client will no longer receive notifications for it. Essentially, it depends on the number of clients registered for that result set. In the following code example, a Continuous Query is unregistered from the cache server.
|
1 |
cache.MessagingService.UnRegisterCQ(cQuery); |
NCache: The Best Solution Out There
When it comes to large and complex business applications, refining data is a huge challenge. NCache addresses this issue with Stream Processing by converting large and complex data into manageable data streams for easy processing.
A popular application of Stream Processing is the Publisher Subscriber model of NCache. However, this comes with the following limitations:
- Messages are not persisted by the applications once they are delivered to the subscribers.
- Data filtration takes place at the client end, making the application architecture more complex.
Continuous Query addresses both of these problems in the following ways:
- By persisting data inside the cache even after processing.
- By filtering data through fairly simple OQL statements at the server end, ensuring a simple application architecture.
Conclusion
NCache is an extremely fast, easy-to-scale, distributed cache that deals with database bottlenecks efficiently and effectively. Just like Continuous Query, Stream Processing, and Pub/Sub Messaging, NCache has many other rich and powerful features that you should try out. These features give you both quantitative and qualitative results. Try NCache now!
