Deploy and Run Fog Rectification for Video on NVIDIA Jetson

This example uses:

This example shows how to generate and deploy a CUDA® executable for a video-based fog rectification application. The example shows the deployable code generation capabilities that the MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms provides for the MATLAB VideoReader function. This example generates a CUDA application that reads the contents of a video file, performs fog rectification operation, and displays the output video on the NVIDIA® hardware.

Prerequisites

Target Board Requirements

NVIDIA Jetson embedded platform.
Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network).
GStreamer and SDL libraries on the target.
Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.
A monitor connected to the display port of the target.

Development Host Requirements

For CUDA code generation, NVIDIA CUDA toolkit on the host and environment variables for the compilers and libraries. For more information, see Third-Party Hardware (GPU Coder) and Setting Up the Prerequisite Products (GPU Coder).

Connect to NVIDIA Jetson

The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.

Create Jetson Object

To communicate with the NVIDIA hardware, create a live hardware connection object by using the jetson function.

hwobj = jetson('jetson-board-name','ubuntu','ubuntu');

When connecting to the target board for the first time,you must provide the host name or IP address, user name, and password of the target board. On subsequent connections, you do not need to supply the address, user name, and password. The hardware object reuses these settings from the most recent successful connection to an NVIDIA board.

By default, this example reuses the settings from the most recent successful connection to a NVIDIA Jetson board.

hwobj = jetson;

Checking for CUDA availability on the Target...
Checking for 'nvcc' in the target system path...
Checking for cuDNN library availability on the Target...
Checking for TensorRT library availability on the Target...
Checking for prerequisite libraries is complete.
Gathering hardware details...
Checking for third-party library availability on the Target...
Gathering hardware details is complete.
 Board name              : NVIDIA Jetson Nano Developer Kit
 CUDA Version            : 10.2
 cuDNN Version           : 8.2
 TensorRT Version        : 8.2
 GStreamer Version       : 1.14.5
 V4L2 Version            : 1.14.2-1
 SDL Version             : 1.2
 OpenCV Version          : 4.1.1
 Available Webcams       :  
 Available GPUs          : NVIDIA Tegra X1
 Available Digital Pins  : 7  11  12  13  15  16  18  19  21  22  23  24  26  29  31  32  33  35  36  37  38  40

During the hardware live object creation, the support package performs hardware and software checks, installs MATLAB IO server on the target board, and gathers information on peripheral devices connected to the target. This information is displayed in the Command Window. In case of a connection failure, a diagnostics error message is reported at the MATLAB command line. If the connection has failed, the most likely cause is incorrect IP address or host name.

Verify GPU Environment on Target Board

To verify that the compilers and libraries necessary for running this example are set up correctly, use the coder.checkGpuInstall (GPU Coder) function.

envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);

The `fog_rectification` Entry-Point Function

The fog_rectification entry-point function takes a foggy video as an input and displays the defogged video.

type fog_rectification.m

function fog_rectification() %#codegen
% Copyright 2019-2021 The MathWorks, Inc.

coder.gpu.kernelfun;
hwobj = jetson();
width = 600;
height = 404;
videoFileName = 'foggy.mp4';
vobj = VideoReader(hwobj,videoFileName,'Width',width,'Height',height);
dispObj = imageDisplay(hwobj);

while vobj.hasFrame
    input = vobj.readFrame;
    
    % Run fog rectification on the frame
    outrectifiedImage = fog_rectification_algorithm (input,width,height);
    reshapedImg = cat(3, outrectifiedImage(:,:,1).', ...
        outrectifiedImage(:,:,2)',outrectifiedImage(:,:,3)');
    image(dispObj,reshapedImg);
end
end

The video file name input accepts full and relative paths. For code generation, provide the video file path on the target. For example, if the video file is available at the location '/home/ubuntu/video/foggy.mp4', then you must provide this path.

Generate CUDA Code for the Target Board Using GPU Coder

To generate a CUDA executable that you can deploy on to an NVIDIA target, create a GPU code configuration object for generating an executable.

cfg = coder.gpuConfig('exe');

To create a configuration object for the Jetson platform and assign it to the Hardware property of the code configuration object cfg, use the coder.hardware function.

cfg.Hardware = coder.hardware('NVIDIA Jetson');

To specify the folder for performing remote build process on the target board, use the BuildDir property. If the specified build folder does not exist on the target board, then the software creates a folder with the given name. If no value is assigned to cfg.Hardware.BuildDir, the remote build process occurs in the last specified build folder. If there is no stored build folder value, the build process takes place in the home folder.

cfg.Hardware.BuildDir = '~/remoteBuildDir';

Set the GenerateExampleMain property to generate an example C++ main file and compile it. This example does not require modifications to the generated main files.

cfg.GenerateExampleMain = 'GenerateCodeAndCompile';

To generate CUDA code, use the codegen function and pass the GPU code configuration and the size of the inputs for fog_rectification entry-point function. After the code generation takes place on the host, the generated files are copied over and built on the target.

codegen('-config ',cfg,'fog_rectification','-report');

Code generation successful: View report

Copy Video File onto Target Board

Move the video file to the target board.

putFile(hwobj,'foggy.mp4', hwobj.workspaceDir);

Run Fog Rectification on Target Board

To run the generated executable on the target board, use the MATLAB runApplication function.

pid = runApplication(hwobj,'fog_rectification');

### Launching the executable on the target...
Executable launched successfully with process ID 10518.
Displaying the simple runtime log for the executable...

Note: For the complete log, run the following command in the MATLAB command window:
system(hwobj,'cat /home/ubuntu/remoteBuildDir/MATLAB_ws/R2024b/home/nrichmon/Documents/MATLAB/ExampleManager/nrichmon.Bdoc24b.FoggyVideo2/nvidia-ex91971069/fog_rectification.log')

A window opens on the target hardware display showing the fog rectification output of the recorded video.