Configuring MapReduce

You can configure MapReduce for processing and generating large data sets with a parallel, distributed algorithm on a cluster.

• Click on the cache name in Cache Explorer to open cache settings.

• Go to the MapReduce tab.

• Click Deploy Task Libraries.

  

• A dialog box will open. Browse for the libraries which have the MapReduce interfaces implemented and click Open.

• A Notification will appear after successful deployment of assemblies.

• Maximum number of MapReduce tasks to be executed simultaneously can be changed according to your requirements.

  

• In case you expect exceptions to be thrown during task execution, you can specify the number of exceptions to be avoided from your code, after which the task is failed and logged in the cache error log.

  

• You can modify chunk size - the number of tasks processed in the Mapper and Combiner - before transmitting to Combiner or Reducer.

  

• You can modify the maximum number of tasks that can wait in queue before they are processed.