Main Content

Classify Images Using TensorFlow Model Predict Block

This example shows how to use the TensorFlow Model Predict block for image classification in Simulink®. The block accepts an image and passes it to a pretrained TensorFlow™ neural network classification model that is executed in Python®. The predicted response for the image is then output to Simulink.

MATLAB® supports the reference implementation of Python, often called CPython. If you use a Mac or Linux® platform, you already have Python installed. If you use Windows®, you need to install a distribution, such as those found at https://www.python.org/downloads/. For more information, see Configure Your System to Use Python. Your MATLAB Python environment must have the tensorflow module installed.

The TensorFlow Model Predict block requires a pretrained TensorFlow model that you saved in SavedModel format or as a Keras HDF5 file in Python. This example provides the saved model digitsdlnetworkwithnoise, which was saved under tensorflow version 2.14.0.

Open Provided Simulink Model

This example provides the Simulink model slexDigitsTFModelPredictExample.slx, which includes the TensorFlow Model Predict block. You can open the Simulink model or create a new model as described in the next section.

Open the Simulink model slexDigitsTFModelPredictExample.slx.

open_system("slexDigitsTFModelPredictExample");

When you open the Simulink model, the software runs the code in the PreLoadFcn callback function before loading the Simulink model. The PreLoadFcn callback function of slexDigitsTFModelPredictExample includes code to check if your workspace contains the inputIms variable for the trained model. If the workspace does not contain the variable, PreLoadFcn loads the image data for the Simulink model. To view the callback function, in the Setup section on the Modeling tab, click Model Settings and select Model Properties. Then, on the Callbacks tab, select the PreLoadFcn callback function in the Model callbacks pane.

Create Simulink Model

To create a new Simulink model, open the Blank Model template and add the TensorFlow Model Predict block from the Deep Learning Toolbox™ library. The block can be configured to accept input data in the form of an h-by-w-by-c-by-N numeric array, where h, w, and c are the height, width, and number of channels of the images, respectively, and N is the number of images.

Insert a Video From Workspace block from the Computer Vision Toolbox™ library and connect it to the input port of the TensorFlow Model Predict block. Double-click the Video From Workspace block to open the Block Parameters dialog box. Specify Signal as inputIms and Form output after final value by as Holding final value. Click OK.

Double-click the TensorFlow Model Predict block to open the Block Parameters dialog box. On the Specify model file tab, specify digitsdlnetworkwithnoise.

The input signal inputIms is a sequence of eight grayscale (one-channel) images that are 28-by-28 pixels in size. The saved digitsdlnetworkwithnoise TensorFlow model expects a four-dimensional signal with dimensions in a different order. On the Inputs tab, under Permutation to Python, specify [4 1 2 3].

Add a To Workspace block to the model and connect it to the output port of the TensorFlow Model Predict block. Double-click the To Workspace block to open the Block Parameters dialog box. Change the Variable name to yPred.

Open the Configuration Parameters dialog box. On the Modeling tab, click Model Settings to open the Configuration Parameters dialog box. On the left of the dialog box, click Solver. Under Simulation time, set Stop time to 7. Under Solver selection, set Type to Fixed-step, and set Solver to discrete (no continuous states). Click OK.

Load the digit sample data as an image datastore. The imageDatastore function automatically labels the images based on the folder names, and stores the data as an ImageDatastore object.

digitDatasetPath = fullfile(matlabroot,"toolbox","nnet","nndemos", ...
    "nndatasets","DigitDataset");
imds = imageDatastore(digitDatasetPath, ...
    IncludeSubfolders=true,LabelSource="foldernames");

Randomly select eight images from the image datastore, create the array of images displayIms, and display the selected images by using the function montage (Image Processing Toolbox).

rng(0,"twister") % For reproducibility
perm = randperm(10000,8);
for i = 1:8
    displayIms(:,:,:,i) = imread(imds.Files{perm(i)});
end
montage(displayIms,size=[1 NaN]);

Figure contains an axes object. The hidden axes object contains an object of type image.

Convert the images in displayIms to single data type.

inputIms = single(displayIms);

Specify the model folder that contains the pretrained network digitsdlnetworkwithnoise in the saved model format. digitsdlnetworkwithnoise can classify images of digits.

if ~exist("digitsdlnetworkwithnoise","dir")
    unzip("digitsdlnetworkwithnoise.zip")
end

Specify the class names.

classNames = {'0','1','2','3','4','5','6','7','8','9'};

Save the model as slexDigitsTFModelPredictExample.slx in Simulink.

Predict Using Simulink Model

Unzip the digitsdlnetworkwithnoise.zip file that contains the TensorFlow model.

unzip("digitsdlnetworkwithnoise.zip")

Simulate the model and save the simulation output to modelOutput. You might receive a warning message if your Python installation uses a tensorflow version prior to 2.14.0. The field modelOutput.yPred.Data contains the classification results.

modelOutput = sim("slexDigitsTFModelPredictExample");

Display the sequence of images and the classification results.

tiledlayout(1,12,TileSpacing="None");
for i = 1:size(inputIms,4)
   nexttile
   imshow(displayIms(:,:,:,i))
   label = modelOutput.yPred.Data(:,:,i)==1;
   title([classNames{label}],FontSize=20)
end

Figure contains 8 axes objects. Hidden axes object 1 with title 8 contains an object of type image. Hidden axes object 2 with title 9 contains an object of type image. Hidden axes object 3 with title 1 contains an object of type image. Hidden axes object 4 with title 9 contains an object of type image. Hidden axes object 5 with title 6 contains an object of type image. Hidden axes object 6 with title 0 contains an object of type image. Hidden axes object 7 with title 2 contains an object of type image. Hidden axes object 8 with title 5 contains an object of type image.

See Also

| | |

Related Topics