Main Content

getCheckpointFile

Class: matlab.compiler.mlspark.RDD
Namespace: matlab.compiler.mlspark

Get the name of the file to which an RDD is checkpointed

expand all in page

Syntax

file = getCheckpointFile(obj)

Description

file = getCheckpointFile(obj) gets the name of the file to which the RDD obj is checkpointed.

Input Arguments

expand all

An input RDD, specified as an RDD object.

Output Arguments

expand all

The name of the file to which an RDD was checkpointed, returned as a character vector.

Examples

expand all

Get the name of the file to which an RDD was checkpointed.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% getCheckpointFile
sc.setCheckpointDir('myDir')  % set the checkpoint directory
x = sc.parallelize({1,2,3})
y = x.map(@(x)({x,1}));
y.checkpoint()  % tell spark to checkpoint the RDD
y.collect()
% need to call collect, so that spark actually materializes the RDD 
% and checkpoints to the myDir directory
y.getCheckpointFile()

Version History

Introduced in R2016b

See Also

| | |