Main Content

Video Send

Send video stream to remote hardware

Since R2021b

expand all in page
  • Simulation 3D Video Send

Libraries:
UAV Toolbox / Simulation 3D

Description

The Video Send block sends video streams from Simulink® to a specified remote device. For hardware-in-the-loop simulation applications, you can send grayscale and RGB images, as well as depth and lidar point cloud data collected from Unreal Engine® scenes to the remote device. You can also specify the compression and quality of the video stream. To stream the video, this block picks the first available local UDP port, independent of the remote port you specify. This block uses the Gstreamer framework to handle data streaming.

Examples

Stream Camera, Depth and Semantic Segmentation Data from Unreal Engine to NVIDIA Jetson

Stream Camera, Depth and Semantic Segmentation Data from Unreal Engine to NVIDIA Jetson

Stream simulated camera, depth, and semantic segmentation label data from an Unreal Engine® scene to NVIDIA® Jetson™ hardware using the Video Send block in Simulink®. It then shows how to visualize incoming data streams on a monitor connected to the Jetson platform, by deploying separate models for each incoming data stream. The deployed models contain the Network Video Receive and SDL Video Display blocks from the MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE® Platforms.

Limitations

  • The Video Send block is supported only for use in Windows and Mac.

Ports

Input

expand all

RGB or grayscale image signal to stream, specified as an M-by-N-by-3 matrix or M-by-N matrix, respectively.

Dependencies

The Image Signal parameter must be set to One multidimensional signal.

Data Types: uint8 | uint16

Red channel signal of the RGB image to stream, specified as an M-by-N matrix.

Dependencies

The Image Signal parameter must be set to Separate color signals.

Data Types: uint8

Green channel signal of the RGB image to stream, specified as an M-by-N matrix.

Dependencies

The Image Signal parameter must be set to Separate color signals.

Data Types: uint8

Blue channel signal of the RGB image to stream, specified as an M-by-N matrix.

Dependencies

The Image Signal parameter must be set to Separate color signals.

Data Types: uint8

Parameters

expand all

Video parameters

Specify the input video stream signal format as one of the following:

  • RGB - RGB image (8-bit per color channel).

  • Grayscale - Grayscale image (8-bit).

  • Grayscale (16-bit) - Grayscale image (16-bit).

Specify the nature of the RGB image input signal as one of the following:

  • One multidimensional signal - One input port for an M-by-N-by-3 color video signal with R, G, and B color channels.

  • Separate color signals - Three separate input ports for R,G and B channels. Each port accepts one M-by-N matrix.

Dependencies

The Format parameter must be set to RGB.

Specify the compression format for the video stream as one of the following:

  • JPEG - Uses jpeg GStreamer plugin.

    • This uses the following GStreamer pipeline: 

      video/x-raw,format=I420 ! jpegenc quality=<Quality> idct-method=1 ! rtpjpegpay ! udpsink sync=false host=<Remote address> port=<Remote port>

  • VP8 - Uses vpx GStreamer plugin.

    • This uses the following GStreamer pipeline: 

      vp8enc deadline=<Max frame time*1000> bits-per-pixel=0.0434 target-bitrate=0 threads=8 lag-in-frames=0 ! rtpvp8pay ! udpsink sync=false host=<Remote address> port=<Remote port>

  • VP9 - Uses vpx GStreamer plugin.

    • This uses the following GStreamer pipeline: 

      vp9enc deadline=<Max frame time*1000> bits-per-pixel=0.0434 target-bitrate=0 threads=8 lag-in-frames=0 ! rtpvp9pay ! udpsink sync=false host=<Remote address> port=<Remote port>

Specify the quality of the JPEG video stream as a positive scalar. This parameter controls the encoding speed and compression ratio of the video stream. A higher quality value increases the image quality at the expense of a higher network bandwidth. Specify a lower value for low-bandwidth network connections.

Dependencies

The Compression parameter must be set to JPEG.

Specify the maximum frame time of the VP8 or VP9 video stream in milliseconds, as a positive integer. This parameter controls the maximum processing time that the codec uses to encode an image. Set this parameter based on the frame rate of the input Image. A good starting point value is 1000/fps, where fps is the sample rate of the input video signal. Set 0 if you want the encoder to take as long as it needs, which will increase the quality at the expense of time required for compression.

Dependencies

The Compression parameter must be set to VP8 or VP9.

Connection parameters

Specify the IP address or host name of the remote device, to which the block sends the message, as a character vector.

Specify the IP port of the remote device, to which the block sends the message. When streaming the video, this block picks the first available local UDP port, independent of the remote port you specify.

Tips

  • VP8 and VP9 compression formats are bandwidth efficient but also computationally expensive. Hence, choose JPEG compression format if you are working on machines with limited CPU resources.

  • To receive the video stream on the remote device, in a separate model, use the Network Video Receive block from the MATLAB® Coder™ Support Package for NVIDIA® Jetson™ and NVIDIA DRIVE® Platforms. Alternatively, you can use your custom GStreamer-based receiver.

Version History

Introduced in R2021b

See Also

| |