Main Content

Classify Objects Using Deep Learning Algorithm on Raspberry Pi Hardware

This example uses:

Open Script

This example shows how to use the Simulink® Support Package for Raspberry Pi® Hardware to deploy a deep learning algorithm that classifies objects using the ResNet-50 convolutional neural network. This pretrained network is 50 layers deep and can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many more. You can experiment with different objects in your surroundings to see how accurately the network classifies images on the Raspberry Pi hardware.

This example uses the Deep Learning template that provides a preconfigured Simulink model to help you design deep learning applications using this support package. You can choose to simulate your model and observe the results in MATLAB® or deploy it on your Raspberry Pi hardware.

Note: You cannot generate and deploy deep learning code on Raspberry Pi hardware using macOS.

Prerequisites

  • Configure the Raspberry Pi network using the Hardware Setup window. During this process, download the Linux® Raspbian image for deep learning. Ensure that you select the option to install the ARM® Compute Library.

  • For more information on how to use the Simulink Support Package for Raspberry Pi Hardware to run a Simulink model on Raspberry Pi hardware, see Get Started with Simulink Support Package for Raspberry Pi Hardware.

Dependencies

  • Simulink Support Package for Raspberry Pi Hardware

  • MATLAB Coder™ Interface for Deep Learning Libraries

  • Deep Learning Toolbox™

  • Deep Learning Toolbox Model for ResNet-50 Network

  • Simulink Coder

  • Computer Vision Toolbox™

  • MATLAB Coder

Required Hardware

  • Raspberry Pi hardware board (Model 4 recommended)

  • Power adapter for the Raspberry Pi board

  • A monitor connected to the Raspberry Pi hardware and a HDMI cable (optional)

  • Supported USB web camera or Raspberry Pi camera module. This example uses a USB web camera.

Hardware Setup

  1. Power the Raspberry Pi target board.

  2. Connect the web camera to the Raspberry Pi target board using the USB port.

Configure Deep Learning Template

You can use either the preconfigured Simulink model or configure the Deep Learning template for this example.

Open the raspberrypi_object_classification Simulink model.

From the Simulink Start Page, in the Simulink Support Package for Raspberry Pi Hardware section, select the Deep Learning template. This template is preconfigured for deploying the deep learning algorithm that targets the latest version of the ARM Compute Library. This example supports the latest version of the ARM Compute Library 20.02.

Deep Learning Template

In the Add Simulation Sources pane, you can provide an input image or its RGB equivalent values to the deep learning algorithm. Connect the Input Source switch from the Design your deep learning algorithm here pane to the Add Simulation Sources pane. Replace the RGB Constant convert to image blocks with the Image From File and Resize blocks, respectively.

Browse an input image file in the Block Parameters dialog box of the Image From File block Parameters dialog box. In this example, the default image is set to peppers.png.

You can change the size of an image or the region of interest (ROI) within an image using the Resize block. The Resize block ensures that the output resolution of the image from the Add Simulation Sources pane matches with the resolution of the images captured from the Raspberry Pi web camera. Configure these parameters in the Block Parameters dialog box of the Resize block.

  1. Set Specify to Number of output rows and columns.

  2. Enter [320 240] in the Number of output rows and columns parameter.

The Raspberry Pi V4L2 Video Capture and convert to image blocks in the Add RaspberryPi Peripherals, Sensors & Inputs pane are preconfigured to their default values.

Double-click the Subsystem block in the Design your deep learning algorithm here pane and build the subsystem using the Image Classifier, To String, String to ASCII, and Insert Text blocks.

The Image Classifier block is used to classify the data using the ResNet-50 neural network. This block predicts class labels for the input image using the trained network imported from a MATLAB function. Configure these parameters in the Block Parameters dialog box of the Image Classifier block.

  1. Set Network to Network from MATLAB function.

  2. Enter resnet50 in the MATLAB function parameter.

The Image Predictor block outputs the predicted class labels with the highest score ypred. This output is then converted to a string value and displayed as a label for the object classified in the input image. The Insert Text block accepts the input image and the Image Predictor block output, which the Insert Text block further converts to ASCII inputs.

The ImagetoRGB MATLAB Function block converts the input image to its equivalent R, G, and B data values.

In the Add Display, Visualization & Outputs pane, you can view the simulation results for the input image or its RGB equivalent values. Add the Display block and connect it to the Label output of the subsystem in the Design your deep learning algorithm here pane. The Display block displays the label of the classified object using the ResNet-50 neural network.

The Raspberry Pi SDL Video Display block in the Add Display, Visualization & Outputs pane is preconfigured to its default values.

Set Parameters in Configuration Parameters Dialog Box

On the Hardware tab of the Simulink model, select Configuration Parameters > Code Generation > Interface. In the Deep Learning section, these parameters are preconfigured for the Deep Learning template.

  1. Set Target library to ARM Compute.

  2. Set ARM Compute Library version to 20.02.1. This example supports the latest version of ARM Compute Library.

  3. Set ARM Compute Library architecture to armv7.

Run Simulink Model

1. On the Hardware tab of the Simulink model, select Configuration Parameters > Hardware Implementation > Target hardware resources > Board Parameters and enter the IP address of the Raspberry Pi hardware board in the Device address parameter.

2. In the Advanced parameters section, ensure that the Use Simulink Coder features option is selected.

3. In the Simulink model, double-click the Input Source switch in the Design your deep learning algorithm pane and connect it to the output in the Add Simulation Sources pane.

4. On the Simulation tab of the model, click Run.

5. Observe that the Display block in the Add Display and Visualization pane displays the label of the object classified in the input image. The SDL Video Display block in the Add RaspberryPi Actuators & Outputs pane displays the input image with the label.

Deploy Simulink Model on Raspberry Pi Target Board

1. Double-click the Input Source switch in the Design your deep learning algorithm pane and connect it to the output in the Add RaspberryPi Peripherals, Sensors & Input pane.

2. On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Build, Deploy & Start.

3. Place an object to be classified in front of the web camera. In this example, a pair of sunglasses is used for object classification.

4. On the Raspberry Pi desktop, the SDL Video Display window opens automatically and displays the input image classified with the label. Observe the sunglass_label label for the classified object displayed in the top left corner of the window. To view the classified image output on the Raspberry Pi desktop, you can use the VNC viewer or connect your monitor to the Raspberry Pi hardware using the HDMI cable.

Other Things to Try

Use a different neural network and observe the object classification label for the input image.