Distributed Cache Continuous Query for Developing Real Time Applications

High traffic real time applications are widely used in the enterprise environment. In real-time applications, information is made available to you in moments it’s produced and any delay in doing so can cause a serious financial loss. The main challenge faced by these high traffic real time applications is to get notified about any changes in data set so that the corresponding views can be updated.

But, these high traffic real time applications cannot rely on the traditional database because they only support queries on the residing data and in order to get the updated data set you have to again execute the query after specific interval which is not instantaneous. And, this periodic polling also causes performance and scalability issues because you are making expensive database trips mostly even when there are no changes in data set.

SqlDependency is provided by Microsoft in SQL Server, and Oracle on Windows also supports it. SqlDependency allows you to specify a SQL statement, and SQL Server monitors this data set in the database for any additions, updates, or deletions and notifies you when this happens. But the problem with SqlDependency is that once it is fired, it gets unregistered from the database. Therefore, all future changes in your data set are missed and you don’t get notified.

Moreover, SqlDependency does not provide details of the record where change occurred. So in order to find the change in data set, you have to fetch the complete data set again instead of directly fetching only the specific record, which is added, updated or removed from data set. And, of course, this is not efficient.

In addition to the limitations of SqlDependency, your database is also not able to cope with the transactional demands of these high traffic real time applications where tens of thousands of queries are executed every second and database quickly becomes a scalability bottleneck. This is because although you can linearly scale your application-tier by adding more application servers, you cannot do the same with your database server.

This is where a distributed cache like NCache comes in because it allows you to cache data and reduce those expensive database trips which are causing scalability bottleneck.

NCache has a powerful Continuous Query capability that enables you to register an SQL-like query with the cache cluster. This Continuous Query remains active in cache cluster and if there is any change in data set of this query, then NCache notifies your real time application. This Continuous Query approach saves you from periodically executing the same expensive query against database to poll.

Here is sample code for NCache Continuous Query:

public void Main(string[] args)
    String query = "SELECT NCacheQuerySample.Business.Product WHERE
                    this.ProductID > 100";

    Hashtable values = new Hashtable();
    values.Add("ProductID", 100);
    onItemAdded = new ContinuousQueryItemAddedCallback(OnQueryItemAdded);
    onItemUpdated = new ContinuousQueryItemUpdatedCallback(OnQueryItemUpdated);
    onItemRemoved = new ContinuousQueryItemRemovedCallback(OnQueryItemRemoved);

    ContinuousQuery query = new ContinuousQuery(queryString, values);
//data set item is removed
void OnQueryItemRemoved(string key){
   Console.WriteLine("Removed key: {0}", key);
//data set item is updated
void OnQueryItemUpdated(string key){
   Console.WriteLine("Updated key: {0}", key);
//data set item is removed
void OnQueryItemAdded(string key){
   Console.WriteLine("Added key: {0}", key);

And, unlike SqlDependency, NCache Continuous Query remains active and doesn’t get unregistered on each change notification. So, your high traffic real time applications continue to be notified across multiple changes.

NCache Continuous Query also provides you the flexibility to be notified separately on ADD, UPDATE, and DELETE. And, you can specify these events at runtime even after creating a Continuous Query, something SqlDependency doesn’t allow you to do. This also reduces events traffic from the cache cluster to your real time application.

In summary, NCache provides you a very powerful event driven Continuous Query that no other database has. And, NCache is also linearly scalable for your high traffic real time applications.

So, download a fully working 60-day trial of NCache Enterprise and try it out for yourself.

Download NCache Trial | NCache Details

Avatar for Iqbal Khan

Leave a Reply

Your email address will not be published. Required fields are marked *