Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English version of the page.

Note: This page has been translated by MathWorks. Click here to see
To view all translated materials including this page, select Country from the country navigator on the bottom of this page.

Incorporate MATLAB Map and Reduce Functions into a Hadoop MapReduce Job

Create a deployable archive of MATLAB® map and reduce functions, and incorporate it into a Hadoop® mapreduce job

To incorporate MATLAB map and reduce functions into a Hadoop mapreduce job, you create a deployable archive from the map and reduce functions and pass the archive as a payload argument to a job submitted to the Hadoop cluster. A deployable archive contains a:

  • mapper function written in MATLAB.

  • reducer function written in MATLAB.

  • MAT-file containing a datastore that describes the structure of the data and variables to be analyzed.

  • Hadoop settings file that identifies the map and reduce functions, the type of data being analyzed, and other configuration details.

For more information, see Workflow to Incorporate MATLAB Map and Reduce Functions into a Hadoop Job

Functions

deploytoolCompile and package functions for external deployment
mccCompile MATLAB functions for deployment
hadoopCompilerPackage MATLAB Compiler programs for deployment against Hadoop clusters as MapReduce programs

Apps

Hadoop CompilerPackage MATLAB programs for deployment to Hadoop clusters as MapReduce programs

Topics

Workflow to Incorporate MATLAB Map and Reduce Functions into a Hadoop Job

Instructions on how to create a deployable archive of MATLAB map and reduce functions and incorporate it into a Hadoop mapreduce job.

Examples

Example Using the Hadoop Compiler App Workflow

Use the Hadoop Compiler app to create a deployable archive of MATLAB map and reduce functions. You can pass the deployable archive as a payload argument to a job submitted to a Hadoop cluster.

Example Using the mcc Command Workflow

Use the mcc command to create a deployable archive of MATLAB map and reduce functions. You can pass the deployable archive as a payload argument to a job submitted to a Hadoop cluster.

Settings

Configuration File for Creating Deployable Archive Using the mcc Command

Create a configuration file that represents the characteristics of the payload to the Hadoop mapreduce job. You will need this file to create a deployable archive using the mcc command.

Related Information

MapReduce (MATLAB)

Datastore (MATLAB)