Classify Images Using TensorFlow Model Predict Block

Open Provided Simulink Model

This example provides the Simulink model slexDigitsTFModelPredictExample.slx, which includes the TensorFlow Model Predict block. You can open the Simulink model or create a new model as described in the next section.

Open the Simulink model slexDigitsTFModelPredictExample.slx.

open_system("slexDigitsTFModelPredictExample");

When you open the Simulink model, the software runs the code in the PreLoadFcn callback function before loading the Simulink model. The PreLoadFcn callback function of slexDigitsTFModelPredictExample includes code to check if your workspace contains the inputIms variable for the trained model. If the workspace does not contain the variable, PreLoadFcn loads the image data for the Simulink model. To view the callback function, in the Setup section on the Modeling tab, click Model Settings and select Model Properties. Then, on the Callbacks tab, select the PreLoadFcn callback function in the Model callbacks pane.

Create Simulink Model

To create a new Simulink model, open the Blank Model template and add the TensorFlow Model Predict block from the Deep Learning Toolbox™ library. The block can be configured to accept input data in the form of an h-by-w-by-c-by-N numeric array, where h, w, and c are the height, width, and number of channels of the images, respectively, and N is the number of images.

Insert a Video From Workspace block from the Computer Vision Toolbox™ library and connect it to the input port of the TensorFlow Model Predict block. Double-click the Video From Workspace block to open the Block Parameters dialog box. Specify Signal as inputIms and Form output after final value by as Holding final value. Click OK.

Double-click the TensorFlow Model Predict block to open the Block Parameters dialog box. On the Specify model file tab, specify digitsdlnetworkwithnoise.

The input signal inputIms is a sequence of eight grayscale (one-channel) images that are 28-by-28 pixels in size. The saved digitsdlnetworkwithnoise TensorFlow model expects a four-dimensional signal with dimensions in a different order. On the Inputs tab, under Permutation to Python, specify [4 1 2 3].

Add a To Workspace block to the model and connect it to the output port of the TensorFlow Model Predict block. Double-click the To Workspace block to open the Block Parameters dialog box. Change the Variable name to yPred.

Open the Configuration Parameters dialog box. On the Modeling tab, click Model Settings to open the Configuration Parameters dialog box. On the left of the dialog box, click Solver. Under Simulation time, set Stop time to 7. Under Solver selection, set Type to Fixed-step, and set Solver to discrete (no continuous states). Click OK.

Load the digit sample data as an image datastore. The imageDatastore function automatically labels the images based on the folder names, and stores the data as an ImageDatastore object.

digitDatasetPath = fullfile(matlabroot,"toolbox","nnet","nndemos", ...
    "nndatasets","DigitDataset");
imds = imageDatastore(digitDatasetPath, ...
    IncludeSubfolders=true,LabelSource="foldernames");

Randomly select eight images from the image datastore, create the array of images displayIms, and display the selected images by using the function montage (Image Processing Toolbox).

rng(0,"twister") % For reproducibility
perm = randperm(10000,8);
for i = 1:8
    displayIms(:,:,:,i) = imread(imds.Files{perm(i)});
end
montage(displayIms,size=[1 NaN]);

Convert the images in displayIms to single data type.

inputIms = single(displayIms);

Specify the model folder that contains the pretrained network digitsdlnetworkwithnoise in the saved model format. digitsdlnetworkwithnoise can classify images of digits.

if ~exist("digitsdlnetworkwithnoise","dir")
    unzip("digitsdlnetworkwithnoise.zip")
end

Specify the class names.

classNames = {'0','1','2','3','4','5','6','7','8','9'};

Save the model as slexDigitsTFModelPredictExample.slx in Simulink.

Predict Using Simulink Model

Unzip the digitsdlnetworkwithnoise.zip file that contains the TensorFlow model.

unzip("digitsdlnetworkwithnoise.zip")

Simulate the model and save the simulation output to modelOutput. You might receive a warning message if your Python installation uses a tensorflow version prior to 2.14.0. The field modelOutput.yPred.Data contains the classification results.

modelOutput = sim("slexDigitsTFModelPredictExample");

Display the sequence of images and the classification results.

tiledlayout(1,12,TileSpacing="None");
for i = 1:size(inputIms,4)
   nexttile
   imshow(displayIms(:,:,:,i))
   label = modelOutput.yPred.Data(:,:,i)==1;
   title([classNames{label}],FontSize=20)
end