Cache Data Dependency on File
File Dependency provides a mechanism to invalidate the cache data in case of non-relational data sources. Here you can associate a file/folder dependency with a cache data item. Whenever that file/folder is removed or modified, NCache will remove its dependent item from the cache. The cache cleanup thread monitors the dependent file/folder for any change at every CleanInterval.
On the cache cleanup thread, NCache triggers dependency in the following scenarios:
- The dependency file is removed/modified.
- The dependency folder is removed/modified.
- The dependency is created in the non-existent file/folder, but this file/folder is created during the cleanup interval.
Similarly, multiple items can depend on a single file. Any change in a file, either by updating the file or deleting, causes the cache to remove the dependent items from the cache. Likewise, an item can create dependency on multiple files in File Dependency.
You can also provide a delay called startAfter in File Dependency which indicates when to start monitoring the dependent file for any change. In this way, NCache will start checking the dependent file after the startAfter time has elapsed.
Cache Data Dependency on File Prerequisites
Important
Make sure that the NCache Service has access to the file path where the file to be dependent on is placed.
Add File Dependency with Cache Data
The CacheItem is a custom class provided by NCache that can be used to add data to the cache and also lets you set additional metadata associated with an object of this class. This metadata defines the properties of the item like dependencies, expirations, and more. The Add method adds a new item in the cache data whereas the Insert method adds a new item with dependency, and if the item already exists in the cache it overwrites the pre-existing properties.
In the following example, the data added in the cache is the content of the file placed at the specified path as a CacheItem with dependency on the file. Any change in the file will result in the expiration of the dependent data present in the cache. The expired items will then be removed from the cache after the CleanInterval.
// Precondition: Cache is already connected
// Specify the file path to add dependency on the file
string filepath = "D:\\Products.csv";
// Getting product from file
Product product = FetchProductFromFile(filepath);
string key = $"Product: {product.ProductID}";
// Create a new cacheItem with product data
var cacheItem = new CacheItem(product);
// Create the dependency on the file with specified path
cacheItem.Dependency = new FileDependency(filepath);
// Add the file data in the cache with dependency on the file
cache.Add(key, cacheItem);
// Precondition: Cache is already connected
// Specify the file path to add dependency on the file
String filepath = "D:\\Products.csv";
// Getting product from file.
Product product = fetchProductFromFile(filepath);
String key = "Product: " + product.getProductId();
// Create a new cacheItem with product data
var cacheItem = new CacheItem(product);
// Create the dependency on the file with specified path
cacheItem.setDependency(new FileDependency(filepath));
// Add the file data in the cache with dependency on the file
cache.add(key, cacheItem);
# Precondition: Cache is already connected
# Generate a unique key for the fileData
key = "FileData"
file_path = "Path/To/File"
# Get the content of the file as string
file_data = cache.get(key, str)
# Check if the file_data is already in the cache
if file_data is None or not file_data:
# Read the contents of the file placed at the path
file = open(file_path, mode='r')
file_data = file.read()
file.close()
# Create a new cache item with the file data
cache_item = ncache.CacheItem(file_data)
# Create File Dependency on the file placed at filePath
cache_item.set_dependency(ncache.FileDependency(file_path))
# Add the file data in the cache with dependency on the file
cache.insert(key, cache_item)
// Precondition: Cache is already connected
// Generate a unique key for the fileData
let key = "FileData";
let filePath = "test.txt";
// Get the content of the file as string
let fileData = this.cache.get(key, String.class);
let blob = new Blob;
// Read the contents of the file placed at the path
if (!(fileData == null)) {
var file = new FileReader();
file.readAsText(blob, filePath);
}
// Create a new cache item with the file data
let cacheItem = new ncache.CacheItem(fileData);
// Create File Dependency on the file placed at filePath
cacheItem.setDependency(new ncache.FileDependency(filePath));
// Add the file data in the cache with dependency on the file
this.cache.insert(key, cacheItem);
Note
To ensure the operation is fail-safe, it is recommended to handle any potential exceptions within your application, as explained in Handling Failures.
Trigger File Dependency with Delay
If you want to add a dependency to an item that is triggered after a particular time, NCache allows you to specify the time interval for that. The Add method adds a new item in the cache whereas the Insert method adds a new item with dependency and if the item already exists in the cache, it overwrites its properties.
In the following example, the data added in the cache is the content of the file placed at the specified path with dependency on the file. The dependency will be triggered after a span of 20 minutes. Any change in the file will result in the expiration of the dependent data present in the cache. The expired items will then be removed from the cache after the CleanInterval.
// Specify the file path to add dependency on the file
string filepath = "D:\\Products.csv";
// Getting product from file
Product product = FetchProductFromFile(filepath);
// Creating a unique key for product
string key = $"Product: {product.ProductID}";
// Create a new cacheItem with product data
var cacheItem = new CacheItem(product);
// Create the dependency on the file with specified path
cacheItem.Dependency = new FileDependency(filepath, DateTime.Now.AddMinutes(20));
// Add the file data in the cache with dependency on the file
cache.Insert(key, cacheItem);
// Specify the file path to add dependency on the file
String filepath = "D:\\Products.csv";
// Getting product from file
Product product = fetchProductFromFile(filepath);
// Creating a unique key for product
String key = "Product: " + product.getProductId();
// Create a new cacheItem with product data
var cacheItem = new CacheItem(product);
// Create the dependency on the file with specified path
cacheItem.setDependency(new FileDependency(filepath, Date.from(Instant.ofEpochSecond(TimeSpan.FromMinutes(20)._ticks))));
// Add the file data in the cache with dependency on the file
cache.insert(key, cacheItem);
# Generate a unique key for the fileData
key = "FileData"
# Specify the file path to add dependency on the file
file_path = "Path/To/File"
# Get the content of the file as string
file_data = cache.get(key, str)
# Check if the file_data is already in the cache
if file_data is None or not file_data:
# Read the contents of the file placed at the path
file = open(file_path, mode='r')
file_data = file.read()
file.close()
# Create a new CacheItem with the file_data
cache_item = ncache.CacheItem(file_data)
# Create the dependency on the file on the specified path
# The dependency starts after 20 minutes
cache_item.set_dependency(ncache.FileDependency(file_path, datetime.now() + timedelta(minutes=20)))
cache.insert(key, cache_item)
// Generate a unique key for the fileData
let key = "FileData";
let filePath = "test.txt";
// Get the content of the file as string
let fileData = this.cache.get(key, String.class);
let blob = new Blob;
// Read the contents of the file placed at the path
if (!(fileData == null)) {
var file = new FileReader();
file.readAsText(blob, filePath);
}
// Create a new cache item with the key
let cacheItem = new ncache.CacheItem(fileData);
// Create delay for dependency
let now = ncache.Calender.getInstance();
now.add(ncache.Calendar.MINUTE, 20);
let delayTime = now.getTime();
// Create File Dependency on the file placed at filePath
// The dependency starts after 20 minutes
cacheItem.setDependency(new ncache.FileDependency(filePath, delayTime));
// Add the file data in the cache with dependency on the file
this.cache.insert(key, cacheItem);
Add File Dependency on Multiple Files
You can also add File Dependency to an item that is dependent on multiple files or folders. This way a single item can be dependent on multiple items using the Add or Insert method. The Add method adds a new item in the cache whereas the Insert method adds a new item with dependency and if the item already exists in the cache, it overwrites its properties.
The following example adds an item with a key that is dependent on two files placed on the specified paths.
// Specify the file paths to add dependency on the file
string orderFilePath = "D:\\Orders.csv";
string orderDetailFilePath = "D:\\OrderDetail.csv";
string[] filePaths = { orderFilePath, orderDetailFilePath };
// Getting order from file
Order order = FetchOrderFromFile(orderFilePath);
string key = $"Order: {order.OrderID}";
// Create a new cacheItem with order data
var cacheItem = new CacheItem(order);
// Create the dependency on the files with specified path
cacheItem.Dependency = new FileDependency(filePaths);
// Add the file data in the cache with dependency on the files
cache.Add(key, cacheItem);
// Specify the file paths to add dependency on the file
String orderFilePath = "D:\\Orders.csv";
String orderDetailFilePath = "D:\\OrderDetail.csv";
ArrayList<String> paths = new ArrayList<>(){};
paths.add(orderFilePath);
paths.add(orderDetailFilePath );
// Create the dependency on the files with specified paths
FileDependency fileDependency = new FileDependency(paths);
// Getting customer from file
Order order = fetchOrderFromFile(orderFilePath);
// Creating a unique key for the customer
String key = "Order:" + order.getOrderDate();
// Create a new cacheItem with customer data
CacheItem cacheItem = new CacheItem(order);
// Create the dependency on the files with specified path
cacheItem.setDependency(fileDependency);
// Add the file data to the cache with dependency on the files
cache.insert(key, cacheItem);
# Get product from database against productId
product = fetch_product_from_db("1001")
# Create a unique cache key for this product
key = "Product:" + product.get_product_id()
# Create a new cache item
cache_item = ncache.CacheItem(product)
# Specify paths of the master files
# Make sure that the network file paths are accessible by NCache Service
file_names = ["\\fileserver1\\ProductList.csv", "\\fileserver1\\OrderList.csv"]
# Adding File Dependency on the specified cache item
cache_item.set_dependency(ncache.FileDependency(file_names))
# Insert the cache item with File Dependency
cache.insert(key, cache_item)
# For successful addition of item with dependency
# Update or remove the file
// Get product from database against productId
let product = this.fetchProductFromDb(this.productId);
// Create a unique cache key for this product
let key = "Product:" + product.ProductID;
// Create a new cache item
let cacheItem = new ncache.CacheItem(product);
// Specify paths of the master files
// Make sure that the network file paths are accessible by NCache Service
let fileNames = ["\\fileserver1\\ProductList.csv", "\\fileserver1\\OrderList.csv"];
// Adding File Dependency on the specified cache item
cacheItem.setDependency(new ncache.FileDependency(fileNames));
// Insert the cache item with File Dependency
this.cache.insert(key, cacheItem);
// For successful addition of item with dependency
// Update or remove the file
Add File Dependency to Existing Cache Items
NCache also provides you with the ease of adding key dependency to an item already present in the cache, without re-inserting it into the cache.
Important
This approach is comparatively less lightweight and more cost-efficient since items are already present in the cache.
Using UpdateAttributes API
This is done through the CacheItemAttribute class, which has the property of Dependency to be set against the CacheItem. The attribute is then set against the existing key of the item, using the UpdateAttributes method of the ICache interface.
The following example adds an item to the cache without dependency and then sets the File Dependency for the item using the the UpdateAttributes method. This doesn't require adding the item again to the cache.
// Specify the file path to add dependency on the file
string filepath = "D:\\Products.csv";
string key = $"Product: 1";
// Create a new cacheItemAttribute
var attribute = new CacheItemAttributes();
// Create the dependency on the file with specified path
attribute.Dependency = new FileDependency(filepath);
// update the cacheItem
cache.UpdateAttributes(key, attribute);
// Specify the file path to add dependency on the file
String filepath = "D:\\Products.csv";
String key = "Product: 1";
// Create a new cacheItemAttribute
CacheItemAttributes attribute = new CacheItemAttributes();
// Create the dependency on the file with specified path
attribute.setDependency(new FileDependency(filepath));
// Update the cacheItem attributes
cache.updateAttributes(key, attribute);
System.out.println("Cache item with key " + key + " updated with file dependency.");
# Generate a unique cache key for file data
key = "FileData"
# Specify the filepath
file_path = "\\FileHost\\ProductList.csp"
# Create a File Dependency where key is dependent on the file placed at file_path
dependency = ncache.FileDependency(file_path)
# Create a cache item attribute for dependency
attribute = ncache.CacheItemAttributes()
attribute.set_dependency(dependency)
# Set the attributes of dependency against the existing cache key
cache.update_attributes(key, attribute)
# Monitor / Verify dependency through PerfMon Counters or Cache API
// Generate a unique cache key for file data
let key = "FileData";
// Get the content of the file as string
// Make sure that the item already exists in the cache
let fileData = this.cache.get(key, String.class);
// Specify the filepath
let filePath = "\\FileHost\\ProductList.csp";
// Create a File Dependency where key is dependent on the file placed at filePath
var dependency = new ncache.FileDependency(filePath);
// Create a cache item attribute for dependency
var attribute = new ncache.CacheItemAttributes();
attribute.setDependency(dependency);
// Set the attributes of dependency against the existing cache key
this.cache.updateAttributes(key, attribute);
// Monitor/Verify dependency through PerfMon Counters or Cache API
Additional Resources
NCache provides sample application for File Dependency on GitHub.
See Also
.NET: Alachisoft.NCache.Runtime.Dependencies namespace.
Java: com.alachisoft.ncache.runtime.dependencies namespace.
Python: ncache.runtime.dependencies class.
Node.js: FileDependency class.
