Classify Static Image Using Deep Learning on Raspberry Pi

Required Hardware

Prerequisite

Configure the Raspberry Pi using the Hardware Setup screen. Customize the OS on your hardware and ensure that you select the option to install the OpenCV Library.

Step 1: Connect the Raspberry Pi Hardware

Tip: Before you start this example, we recommend you to complete the Getting Started with MATLAB Support Package for Raspberry Pi Hardware and Send Inputs to MATLAB Function from Command Line of Raspberry Pi examples.

Connect the micro end of the USB cable to the Raspberry Pi and the regular end of the USB cable to the computer. Wait until the PWR LED on the hardware starts blinking.

In the MATLAB® Command Window, create a connection to the Raspberry Pi hardware.

r = raspi('rpi4b040307eth');

Step 2: Open the Object Classification Algorithm

In this example, the raspi_fileRead_resnet function loads the ResNet-50 deep learning neural network to classify the input image. The function resizes the image using OpenCV to fit the size expected by the network. It then classifies the image into object categories and outputs a label for object predictions along with the probabilities for each of the object categories.

For more information on the function, enter this command in the MATLAB Command Window.

type raspi_fileRead_resnet

function raspi_fileRead_resnet(fileName)
%#codegen
assert(size(fileName,2)==1); % Filename is a 1d array of size Nx1
assert(all(size(fileName) <= 50));
assert(isa(fileName,'char'));

%Initialize DNN and the input size
net        = coder.loadDeepLearningNetwork('resnet50');
inputSize  = [224, 224,3]; %net.Layers(1).InputSize;

%Read the image
imgSizeAdjusted = readResizedImg(fileName,inputSize);

%Classify the input image
[label,score] = net.classify(imgSizeAdjusted);

labelStr  = cellstr(label);
maxScore  = max(score);
outputStr = sprintf('Label : %s \nScore : %f',labelStr{:},maxScore);

%Print label and score to stdout
fprintf('%s\n',outputStr);
end

You can modify the function by using the edit command.

edit raspi_fileRead_resnet

Step 3: Generate C++ Code for Image Classification on Raspberry Pi

Create a hardware configuration object by using the targetHardware function in the MATLAB Command Window.

board = targetHardware('Raspberry Pi')

board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: 'rpi4b040307eth'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

Set the DeviceAddress.

board.DeviceAddress='rpi4b040307eth';
board.Username = 'pi';
board.Password = 'raspberry';
board.BuildDir = '/home/pi';

Verify the DeviceAddress, Username, and Password properties listed in the output. If required, change the value of the properties by using the dot notation syntax. For example, to change the device address to 172.18.186.123, enter,

board.DeviceAddress = '172.18.186.123';

Set the CoderConfig.GenCodeOnly property of the board object to true. This enables the support package to generate and run the image classification algorithm code only on the host computer.

board.CoderConfig.TargetLang = 'C++';
board.CoderConfig.GenCodeOnly = true;

Create deep learning configuration object. Specify the TargetLibrary to 'none' to generate library free code.

dlcfg = coder.DeepLearningConfig(TargetLibrary = 'none');

You can use CodeReplacementLibrary libraries for improved run-time execution. To utilize "ARM CORTEX-A CMSIS CRL", please make sure you have installed the following:

You can run the following command in the command window:

board.CoderConfig.CodeReplacementLibrary = "ARM Cortex-A CMSIS"; 

Set the DeepLearningConfig property of the code generation configuration object to the deep learning configuration object.

board.CoderConfig.DeepLearningConfig = dlcfg;

Deploy the function on the hardware by using the deploy function. The deploy function initiates code generation of the raspi_fileRead_resnet function. Once code generation is complete, MATLAB generates a code generation report. Use this report to debug the function for any errors and warnings in the generated code.

After successfully generating the code, update the main.cpp file to accept command line arguments.

deploy(board,'raspi_fileRead_resnet');

Code generation successful: View report

Step 4: Edit C++ Main File to Accept Input from Command Line of Raspberry Pi

This section explains how to modify the main.cpp file to provide inputs using the command line of Raspberry Pi. A premodified main.cpp file is available for you to use. You can choose to use the premodified file or to update the file manually using the steps in this section.

To use the premodified main.cpp and mail.h files, enter this command in the MATLAB Command Window.

copyfile('main.h',fullfile(pwd,'codegen','exe','raspi_fileRead_resnet'),'f')
copyfile('main.cpp',fullfile(pwd,'codegen','exe','raspi_fileRead_resnet'),'f')

To update the file manually, copy the generated main.cpp and main.h files to one level higher than the current level in the file hierarchy. Open the main.cpp file from the GENERATED CODE section of the code generation report and update the file by adding code at the points labeled in this code.

Open the main.h file from the GENERATED CODE section of the code generation report and update the file by adding code at the points labeled in this code.

Step 5: Deploy the Function as a Standalone Executable on the Raspberry Pi

To generate and deploy the code on the hardware, set the CoderConfig.GenCodeOnly property of the board object to false and then use the deploy function. Since the deploy function overwrites the main.cpp file that you edited in the previous section, disable the creation of the file before deployment.

board.CoderConfig.CustomSource = {fullfile(pwd,'codegen','exe','raspi_fileRead_resnet','main.cpp')}

board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: 'rpi4b040307eth'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

board.CoderConfig.CustomInclude = {fullfile(pwd,'codegen','exe','raspi_fileRead_resnet')}

board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: 'rpi4b040307eth'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

board.CoderConfig.GenerateExampleMain = 'DoNotGenerate';
board.CoderConfig.GenCodeOnly = false

board = 
targetHardware with properties:

               Name: 'Raspberry Pi'
      DeviceAddress: 'rpi4b040307eth'
           Username: 'pi'
           Password: '*********'
           BuildDir: '/home/pi'
    EnableRunOnBoot: 0
        BuildAction: 'Build, load, and run'
        CoderConfig: [1×1 coder.CodeConfig]

deploy(board,'raspi_fileRead_resnet')

Code generation successful: View report

The deploy function initiates code generation of the raspi_fileRead_resnet function.

After successfully generating the code, the support package loads the object classification algorithm as a standalone executable on the hardware. The location of the executable is displayed in the MATLAB Command Window. Note the location for later use.

Step 6: Copy Image File to the Raspberry Pi

In this example, coffeemug.png is the image input to the executable and the location of the executable is same as the one you noted in the previous step.

4. You can provide the input image of your choice and copy it to the location of the generated elf using the following syntax:

r.putFile('<input image>','<target location of the generated elf copied from the code generation report>')

In this example, coffeemug.png is the image input to the executable and the location of the executable is /home/pi/MATLAB_ws_R2021a/C/Users/Documents/MATLAB/Examples/rasp.

r.putFile('coffeemug.png','/home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/rasp')

Step 7: Run the Executable Program to Get Prediction Scores

For MATLAB Desktop, use this command in the MATLAB Command Window to open an SSH terminal with Raspberry Pi, .

openShell(r)

For MATLAB Online, open an SSH terminal with Raspberry Pi. You can use Putty to connect to Hardware or directly use SSH commands in terminal for Unix.

In the terminal, log in by entering the user name and password for the Raspberry Pi.

Change the current directory to the directory where the executable was saved.

cd /home/pi/MATLAB_ws/R2021a/C/Users/Documents/MATLAB/Examples/rasp

Run the executable on the Raspberry Pi by providing the coffee.png image as an input to the executable.

sudo ./raspi_fileRead_resnet.elf coffeemug.png

The executable starts classifying the object in the image file, then it predicts the object and displays its label and the probability associated with the object category.

See Also