Main Content

aggregate

Class: matlab.compiler.mlspark.RDD
Namespace: matlab.compiler.mlspark

Aggregate the elements of each partition and subsequently the results for all partitions into a single value

expand all in page

Syntax

result = aggregate(obj,zeroValue,seqOp,combOp)

Description

result = aggregate(obj,zeroValue,seqOp,combOp) aggregates the elements into a single value using given combine functions specified by seqOp and combOp, and a neutral “zero value” specified by zeroValue.

Input Arguments

expand all

An input RDD, specified as a RDD object.

A neutral “zero value”, specified as a cell array of numbers.

Data Types: cell

A function to aggregate the values of each key, specified as a function handle.

Data Types: function_handle

A function to aggregate results of seqOp, specified as a function handle.

Data Types: function_handle

Output Arguments

expand all

An RDD containing aggregated elements, returned as an RDD object.

Examples

expand all

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% Aggregate
seqOp = @(x,y)({x{1} + y, x{2} + 1});
combOp = @(x,y)({x{1} + y{1}, x{2} + y{2}});
x = sc.parallelize({1, 2, 3, 4});
y = x.aggregate({0, 0}, seqOp, combOp) % {10,4}

Version History

Introduced in R2016b

See Also

|