Implement Connected I/O on Raspberry Pi Hardware and Publish Data to WebSocket Server

This example uses:

Simulink Support Package for Raspberry Pi Hardware Simulink Support Package for Raspberry Pi Hardware

This example shows how to use the Simulink® Support Package for Raspberry Pi® Hardware to develop an image processing system on Raspberry Pi board using a camera in the connected I/O mode. The example also shows you how to publish the output on a web browser and on a SDL video display screen.

Introduction

In this example, the video captured by a camera is processed using an image processing algorithm implemented on the Raspberry Pi hardware. The connected I/O capabilities are used to establish communication with the camera before deploying the model on the hardware. The image processing algorithm comprises of converting the video to obtain either a red, green, blue, grayscale, edge-detection, or median filtered output. The video processed from the Raspberry Pi hardware can be viewed on a web browser and a SDL video display screen. The former option uses a WebSocket client on the host computer to read video data.

Prerequisites

We recommended that you complete the Get Started with Simulink Support Package for Raspberry Pi Hardware example.

Required Hardware

Raspberry Pi board
Micro USB cable
You can either use a USB camera or the on-board camera. In this example, an external USB camera is interfaced with the Raspberry Pi board.

Task 1: Hardware Setup

Connect the Raspberry Pi board to the host computer.
Connect the USB camera to the Raspberry Pi board.

Task 2: Configure Simulink Model

This support package provides a preconfigured model that you can use to deploy the image processing algorithm on the real-time video captured by the camera.

Open the raspberrypi_image_processing_web_display Simulink model.

This Simulink model has the following areas:

A. Camera output

You can configure the following parameters in the V4L2 Video Capture block in the Camera output area:

Enter the camera identifier in the Device name parameter.
Specify the width in pixels and height in lines of the video in the Image size parameter.
Set the Pixel format parameter to RGB.
Set the Sample time parameter.

B. Algorithm control

The Algorithm Control area allows you to select the type of control, web or manual, to furnish the processed video output through the SDL Display window and the web browser.

Option 0: Convert the video to display only the red component
Option 1: Convert the video to display only the green component
Option 2: Convert the video to display only the blue component
Option 3: Convert the video to display only the grayscale output
Option 4: Convert the video to display only the edge-detection output
Option 5: Filter median noise

You can configure the following parameters in the Manual Control block to channel the processed video output to be displayed on the SDL video display screen:

Set a Constant value that corresponds to the image processing operation.
The default Sample time is 0.5 seconds.

The Web Control subsystem enables you to select the image processing operation from the drop-down menu of the host web page.

Configure the following parameters in the WebSocket subscribe block to channel the processed video output to display on the web browser. Ensure to configure the same Server IP address and the Server port in the WebSocket publish block in the Web browser display area.

Enter the Server IP Address (IP address of the Raspberry Pi board) of the WebSocket server on which the JSON data is hosted. This example uses 192.168.0.111 because we will access the server over a private network.
Enter the Server port of the WebSocket server that accepts the signals from the WebSocket publish block. This parameter is set to 9137 in this example.
Specify the source to obtain the properties of the JSON data in the Specify output info via parameter. This example uses the default value.
Set the Number of outputs parameter to 1.
Specify the Data Size and the Data Type for the Output signal.
Set the Sample time parameter, so that the video data is published on the web browser based on this sample time.

C. Image processing subsystem

The Image processing subsystem receives four inputs.

Red component data of the video captured by the camera
Green component data of the video captured by the camera
Blue component data of the video captured by the camera
Algorithm control subsystem selection output for the desired image processing algorithm

D. Output display

Configure the following parameter in the SDL Video Display block of the Output display area:

Set the Pixel format parameter to RGB.

E. Web browser display

This subsystem receives the processed video output to publish the video data to a WebSocket server and read on a web browser in a WebSocket client.

Configure the following parameters in the WebSocket publish block:

Enter a unique label in the Block label parameter to identify the data signal in the server. This parameter is set to CameraData in this example.
Enter the Server IP address of the server to which you want to publish the video data. This example uses 192.168.0.111 because we will access the server over a private network.
Enter the Server port of the server that accepts the signals from the WebSocket publish block. The server port is set to 9137 in this example.
Set the Number of Inputs parameter to 1.

To interface the external USB camera with the Raspberry Pi board, you must enable the configure these properties before deploying the Simulink model on the hardware.

Open the Simulink model.
On the Modeling tab, select Model Settings.
In the Configuration Parameters dialog box, select Hardware Implementation.
Set the Hardware board to Raspberry Pi. This selection populates the Hardware board settings parameters with the default values for Raspberry Pi hardware.
Set the Communication Interface parameter to TCP/IP.

Task 3: Run Simulink Model to View Processed Video on SDL Video Display

The SDL video display screen is used to view the processed video. The selected processing option can be viewed on the SDL video display screen only when the manual switch selection is set to Manual Control.

Open the Simulink model.
Set the image processing operation option in the Constant parameter of the Manual Control block.
On the Hardware tab of the Simulink model, in the Mode section, select Connected IO and then click Run with IO.
Observe the corresponding video output on the SDL Video Display window.
While the model is simulating, you can change the option in the Constant parameter and observe the video output that corresponds to all the processed operations from 0 through 5.

SDL Video Display output for Option 0: Display only the red component

SDL Video Display output for Option 1: Display only the green component

SDL Display screen output for Option 2: Display only the blue component

SDL Display screen output for Option 3: Display the grayscale output

SDL Display screen output for Option 4: Display the edge-detection output

SDL Display screen output for Option 5: Filter median noise

Task 5: Run Simulink Model to View Processed Video on Web Browser

The web browser displays the selected output for the processed video. The selected processing option can be viewed on the web browser only when the manual switch selection is to Web Control.

Open the Simulink model.
On the Simulation tab, click Run.
To access the web browser dashboard and open on the web browser, enter the following command at the MATLAB® Command Window:

4. Enter the IP Address, WebSocket port, and Camera resolution on the web browser dashboard and click Connect.

Note: The Raspberry Pi IP address specified in the WebSocket publish, WebSocket subscribe, and Raspberry Pi Camera Live Feed dashboard must be the same.

The Raspberry Pi Camera Live Feed dashboard is preconfigured using JavaScript® to read the JSON data from the IP address 192.168.0.111 and the port 9137. Press F12 to view the source code of the HTML page, which explains how the dashboard has been built using JavaScript.

While the model is still simulating, you can select from the drop-down options in the Camera mode menu and observe the processed video output on the web browser dashboard.

Task 6: Deploy Simulink Model on Raspberry Pi Hardware

You can also deploy the Simulink model on the Raspberry Pi hardware to create a standalone system. The drop-down menu on the dashboard of the web browser is used to select from the processed video.

Open the Simulink model.
Select the Control manual switch to Web Control.
On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Build, Deploy & Start.
Enter the following command at the MATLAB Command Window:

5. Enter the IP Address, WebSocket port, and Camera resolution on the web browser dashboard and click Connect.