Main Content

Customize Color Within Video Using Raspberry Pi

This example uses:

Open Model

This example shows you how to replace a particular color in a live video stream with an image on Raspberry Pi® hardware by using a MATLAB Function block.

The Simulink® model in this example compares each pixel in the acquired image with the specified color. If the pixel of the acquired image is close enough to the specified color, the pixel is replaced with the corresponding pixel of the image. This process is known as chroma keying.

In this example, you will learn how to:

  • Acquire live video stream from a webcam or a camera module connected to the Raspberry Pi hardware.

  • Implement the chroma keying algorithm on the acquired video using a MATLAB Function block.

  • Display the processed video on the Raspberry Pi hardware.

Required Products

Required Hardware

  • Raspberry Pi hardware (Model 3B or 3B+ recommended)

  • A supported USB webcam or a Raspberry Pi camera module

  • USB cable

  • Ethernet cable

  • A monitor connected to the Raspberry Pi hardware and an HDMI cable (optional)

Step 1: Connect Raspberry Pi Hardware for Chroma Key

Tip: Before you start this example, we recommend you complete the Get Started with Simulink Support Package for Raspberry Pi Hardware example.

1. Connect the micro-USB end of the USB cable to the Raspberry Pi and the regular USB end of the USB cable to the computer. Wait until the PWR LED on the hardware starts blinking. 2. To connect a video device to the hardware, do any one of the following:

  • If you have a webcam, connect the webcam to one of the USB ports on the hardware. Note that some webcams draw too much power and may require a powered USB hub for proper operation.

  • If you have a Raspberry Pi camera module, connect the camera module to the hardware by using a CSI cable. For information on how to connect the camera module to the hardware, follow the instructions in the manual provided by the manufacturer.

3. Using the Hardware Setup screen, configure the Raspberry Pi network.

Step 2: Open and Configure Chroma Key Simulink Model

Open the raspberrypi_chromakey Simulink model.

The model consists of these blocks:

  • Red, Green, and Blue References: The Constant value in these blocks represent the red (R), green (G), and blue (B) components of the color to be replaced. For example, if the color to be replaced is green, specify the values as 0, 128, and 0. You can specify any other color by changing the values in the Constant value parameter. To find the RGB values of a color, perform these steps:

1. In the MATLAB® Command Window, create a connection to the Raspberry Pi hardware using raspi.

mypi = raspi;

2. Create a connection to the cameraboard and assign the connection to a handle, mycam.

mycam = cameraboard(mypi);

3. Plot and display the image captured in the MATLAB Figure Window.

img = snapshot(mycam);
imshow(img);

4. Place the cursor on the color to be replaced. The MATLAB Data Cursor tool displays the RGB values of the color. Copy these values and paste them in the Constant value parameter of the corresponding block parameters dialog box.

  • V4L2 Video Capture: This block captures live video from the video device connected to the hardware. The block outputs RGB values of the acquired pixel as an 8-bit value.

Double-click the block to specify the name of the video device in the Device name parameter. To find the name of the device:

1. Open the hardware terminal and use the ls command. This command lists all the video devices connected to the hardware.

ls -al /dev/video*

2. Copy the name of your video acquiring device and paste the name in the Device name parameter.

  • MATLAB Function: This block implements the chroma keying algorithm on the acquired video. Chroma keying converts the RGB values of the acquired color to its equivalent Y, Cb, and Cr components. The Y component represents the luminance component and determines how light or dark the video is. Cb and Cr components represent the chroma components to measure similarity to a source color. Measuring color similarity using only the Cb and Cr components makes chroma keying robust to variations in luminance values in light and dark areas of the solid background color. If the measured similarity is within the threshold limit, each pixel in the acquired image is replaced with the corresponding pixel of the image. For information on how chroma keying is implemented in MATLAB, double-click the block.

Step 3: Change Model Configuration Parameters Settings

1. To open the Model Configuration Parameters dialog box, click the gear icon on the Simulink model toolbar.

2. In the Configuration Parameters dialog box, select Hardware Implementation.

3. Set the Hardware board parameter to Raspberry Pi. This selection automatically populates the parameters in the Hardware board settings with the default values for Raspberry Pi.

4. In the Hardware board settings pane, expand Target hardware resources and select Board Parameters. Specify these parameter values:

  • Device Address: The IP address or host name of the hardware.

  • Username: Specify the root user name of the Linux® system running on the hardware. The default user name of the Raspbian Linux distribution is pi.

  • Password: Specify the root password of the Linux system running on the hardware. The default password of the Raspbian Linux distribution is raspberry.

5. Click Apply. Click OK to save your changes.

Step 4: Signal Monitoring and Parameter Tuning

On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Monitor & Tune. The lower-left corner of the model window displays status while the support package generates the code. After successfully generating the code, the support package loads and runs the code on the computer. The executable starts acquiring live video stream from the video device, replaces the green color in the acquired video with the sample image, and then displays the result on the computer.

Step 5: Deploy Chroma Key Model on Raspberry Pi Hardware

On the Hardware tab of the Simulink model, in the Mode section, select Run on board and then click Build, Deploy & Start. The lower-left corner of the model window displays status while the support package generates the code. After successfully generating the code, the support package loads and runs the code on the hardware. The executable starts acquiring live images from the video device, replaces the green color in the acquired video with the sample image, and then displays the result on the hardware screen. To view the Raspberry Pi screen, use a VNC viewer and perform a remote session on the hardware to get the display. You can alternatively connect an HDMI cable between a monitor and the hardware.

Other Things to Try

Modify the color to replace and deploy the model on the Raspberry Pi hardware again.

See Also

Detect Boundaries of Objects Within Video Using Raspberry Pi