Implement MapReduce Interface

Note

This feature is only available in NCache Enterprise Edition.

Users can implement the following interfaces:

Pre-Requisites

In order to implement MapReduce interface, include the following namespaces in your application:
- Alachisoft.NCache.Runtime.Mapreduce
- Alachisoft.NCache.Runtime.Caching
Make sure that the classes are marked as serializable.
This should be a class library project using Microsoft Visual Studio.
All assemblies including dependent assemblies must be deployed.

Mapper

Implement the interface IMapper and provide implementation for Map() method. This method will contain the logic to map the input from cache into more meaningful and goal specific key-value pairs which can be sent to the Reducer or optional Combiner. Referring to the workflow diagram, the string input is transformed by emitting each word with a key-value pair like <word, count> in the Mapper.

[Serializable]
public class ProductCountMapper : IMapper
{

    public void Map(object key, object value, IOutputMap context)
    {
        if (value is IList<string>)
        {
            IList<string> order = value as IList<string>;

            foreach (var product in order)
            {
                context.Emit(product, 1);
            }
        }
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Combiner Factory

Implement the interface ICombinerFactory and provide implementation for its Create(key) method. Providing Combiner Factory or Combiner is optional. This method will provide the incoming element with a new instance of the Combiner so that it merges the intermediate key-value pairs from the Mapper.

[Serializable]
public class ProductCountCombinerFactory : ICombinerFactory
{
    public IDictionary<string, ICombiner> combiners = new Dictionary<string, ICombiner>(StringComparer.CurrentCultureIgnoreCase);

    public ICombiner Create(object key)
    {
        if (!combiners.ContainsKey(key as string))
        {
            combiners.Add(key as string, new ProductCountCombiner());
        }

        return combiners[key as string];
    }
}

Combiner

Implement the interface ICombiner. This method reduces the task result locally in chunks so that the Reducer is not burdened with excessive processing. If the workflow diagram is referred, it means that the elements with the same keys are grouped together before being sent to the Reducer for counting the word occurrence.

[Serializable]
public class ProductCountCombiner : ICombiner
{
    int count;

    public void BeginCombine()
    {
        //Any initialization related tasks

        count = 0;
    }

    public void Combine(object value)
    {
        if (value is int)
        {
            count += (int)value;
        }
    }

    public object FinishChunk()
    {
        // Finalize current chunk and 
        // re -initialize resources here

        int countToBeReturned = count;
        count = 0;
        return countToBeReturned;
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Reducer Factory

Implement the interface IReducerFactory and provide implementation for its Create(key) method. Providing Reducer Factory or Reducer is optional. This method will provide the incoming element with a new instance of the Reducer so that it merges the intermediate key-value pairs from the Combiner.

[Serializable]
public class ProductCountReducerFactory : IReducerFactory
{
    IDictionary<string, IReducer> reducers = new Dictionary<string, IReducer>(StringComparer.CurrentCultureIgnoreCase);

    public IReducer Create(object key)
    {
        if (!reducers.ContainsKey(key as string))
        {
            // If you want to map output against a different key,
            // specify a different key in constructor

            reducers.Add(key as string, new ProductCountReducer(key as string));
        }

        return reducers[key as string];
    }
}

Reducer

Implement the interface IReducer. This method will reduce (process) the intermediate key-value pairs into further meaningful pairs. If the workflow diagram is referred, it means that the values of the grouped elements from Combiner are summed up to find the actual word count.

[Serializable]
public class ProductCountReducer : IReducer
{
    int count;
    string outputKey;
    KeyValuePair outputMapEntry;

    public ProductCountReducer(string key)
    {
        outputKey = key;
    }

    public void BeginReduce()
    {
        //All initialization related tasks

        count = 0;

        outputMapEntry = new KeyValuePair();

        // If you want to change keys in output map,
        // Change outputKey to your desired key

        outputMapEntry.Key = outputKey;
    }

    public void Reduce(object value)
    {
        if (value is int)
        {
            count += (int)value;
        }
    }

    public KeyValuePair FinishReduce()
    {
        outputMapEntry.Value = count;

        return outputMapEntry;
    }

    public void Dispose()
    {
        // Dispose resources
    }
}

Key Filter

Implement the interface IKeyFilter to provide implementation for the FilterKey() method. Providing this interface is optional. This method will allow user to filter the input for Mapper by specifying the keys to be processed. If this option is not utilized, the whole cache data will be taken as input for the MapReduce tasks.

[Serializable]
public class ProductCountKeyFilter : IKeyFilter
{
    public bool FilterKey(object key)
    {
        // Execute map for keys containing word "Order"

        return (key as string).Contains("Order");
    }
}

Important

Make sure to deploy the MapReduce task libraries after implementing the interfaces using NCache Web Manager as guided in Configuring MapReduce in Administrators' Guide.