Computer Vision with Simulink

Simulink^® support for computer vision applications

Use Computer Vision Toolbox™ blocks to build models for computer vision applications. Perform feature detection, image analysis, FIR filtering, frequency and Hough transforms, morphology, contrast enhancement, and noise removal.

Local features and their descriptors are the building blocks of many computer vision algorithms. Their applications include image registration, object detection and classification, tracking, and motion estimation.

Motion estimation and tracking are key activities in applications including activity recognition, traffic monitoring, automotive safety, and surveillance.

Analysis and enhancement techniques enable you to increase the signal-to-noise ratio and accentuate features.

The showvipblockdatatypetable function provides details regarding block capabilities, limitations pertaining to code generation, variable-sizing, and supported data types for all Computer Vision Toolbox blocks.

Blocks

expand all

Feature Detection and Recognition

Corner Detection	Calculate corner metric matrix and find corners in images
Edge Detection	Find edges of objects in images using Sobel, Prewitt, Roberts, or Canny method
Trace Boundary	Trace object boundaries in binary images
Template Matching	Locate a template in an image
Estimate Geometric Transformation	Estimate geometric transformation from matching point pairs
Find Local Maxima	Find local maxima in matrices
Template Matching	Locate a template in an image
Warp	Apply projective or affine transformation
Resize	Enlarge or shrink entire image or region of interest within image
Rotate	Rotate image by specified angle
Shear	Shift rows or columns of an image or a video frame by linearly varying offset
Translate	Translate image or video frame

Object Detection

Deep Learning Object Detector

Detect objects using trained deep learning object detector (Since R2021b)

Tracking and Motion Estimation

Block Matching	Estimate motion between images or video frames
Optical Flow	Estimate object velocities
Template Matching	Locate a template in an image

Analysis and Statistics

2-D Autocorrelation	2-D autocorrelation of input matrix
2-D Correlation	Compute 2-D correlation of two input matrices
2-D Histogram	Generate histogram from input
2-D Maximum	Compute maximum value of input or sequence of inputs
2-D Mean	Find 2-D mean of input array
2-D Median	2-D Median values of input array
2-D Minimum	Find minimum values in input or sequence of inputs
2-D Standard Deviation	Compute standard deviation of input or sequence of inputs
2-D Variance	Compute variance of input or sequence of inputs
Blob Analysis	Statistics for labeled regions
Find Local Maxima	Find local maxima in matrices
PSNR	Compute peak signal-to-noise ratio (PSNR) between images

Morphological Operations

Bottom-hat	Perform morphological bottom-hat filtering on intensity or binary images
Closing	Perform morphological closing on binary or intensity images
Dilation	Dilate binary or intensity image by finding local maxima
Erosion	Find local minima in binary or intensity image
Label	Label connected components in binary image
Opening	Perform morphological opening on binary or intensity images
Top-hat	Perform morphological top-hat filtering on intensity or binary images

Color Space Formatting and Conversions

Autothreshold	Convert intensity image to binary image
Chroma Resampling	Downsample or upsample chrominance components of images
Color Space Conversion	Convert color space of image
Demosaic	Demosaic Bayer format images
Gamma Correction	Apply or remove gamma correction to or from image or video stream
Image Complement	Compute the complement of pixel values in binary or intensity images
Image Data Type Conversion	Convert and scale input image to specified output data type
Image Pad	Pad image by adding rows, columns, or both
To Simulink Image	Pack numeric matrix into a Simulink image (Since R2022a)
From Simulink Image	Unpack numeric matrix from Simulink image (Since R2022a)
Image Attributes	Output attributes of Simulink image signal (Since R2022b)

Filters, Transforms, and Enhancements

2-D Convolution	Compute 2-D discrete convolution of two input matrices
2-D FFT	Compute 2-D fast Fourier transform (FFT)
2-D IFFT	Compute 2-D inverse fast Fourier transform (IFFT)
2-D DCT	Compute 2-D discrete cosine transform (DCT)
2-D IDCT	Compute 2-D inverse discrete cosine transform (IDCT)
2-D FIR Filter	2-D FIR filter on input matrix
Block Matching	Estimate motion between images or video frames
Block Processing	Repeat user-specified operation on blocks of input matrix
Contrast Adjustment	Adjust image contrast using linear scaling
Deinterlacing	Remove interlacing effect
Edge Detection	Find edges of objects in images using Sobel, Prewitt, Roberts, or Canny method
Histogram Equalization	Enhance contrast of images using histogram equalization
Median Filter	Perform 2-D median filtering
Hough Transform	Find lines in images
Hough Lines	Find Cartesian coordinates of lines described by rho and theta pairs
Gaussian Pyramid	Perform Gaussian pyramid decomposition

Video Loading, Saving, and Display

Write Binary File	Write binary video data to file
Image From File	Read image from file location
Image From Workspace	Import image from MATLAB workspace
Video Viewer	Display images or video frames
From Multimedia File	Read video frames and audio samples from multimedia file
To Multimedia File	Write video frames and audio samples to multimedia file
To Video Display	Display images or video frames
Frame Rate Display	Calculate and display video frame rate
Video To Workspace	Export image or video to MATLAB workspace
Video From Workspace	Import video from MATLAB workspace
Read Binary File	Read video data from binary file

Graphics

Compositing	Combine two images or apply mask to image
Draw Markers	Draw markers on image
Draw Shapes	Draw rectangles, lines, polygons, or circles on images
Image Pad	Pad image by adding rows, columns, or both
Insert Text	Draw text on images or video frames

Visualize Point Clouds

Point Cloud Viewer

Visualize streaming point cloud data sequence (Since R2023a)

Objects

Simulink.ImageType Specify image data type (Since R2021b)

Topics

Video Formats
Video data is a series of images over time.
Image Formats
In the Computer Vision Toolbox software, images are real-valued ordered sets of color or intensity data.
Fixed-Point Signal Processing
Discusses advantages of fixed-point development in general and of fixed-point support in System Toolbox software in particular, as well as lists common applications of fixed-point signal processing development.
Fixed-Point Concepts and Terminology
Defines fixed-point concepts and terminology that are helpful to know as you use DSP System Toolbox™ software.
Arithmetic Operations
Describes the arithmetic operations used by fixed-point DSP System Toolbox blocks, including operations and casts that might invoke rounding and overflow handling methods.
Fixed-Point Support for MATLAB System Objects
Enable fixed-point support for Computer Vision Toolbox system objects.
Specify Fixed-Point Attributes for Blocks (DSP System Toolbox)
Teaches you how to specify fixed-point attributes and parameters in software on both the block and system levels.
Visualize Point Cloud Sequence
This example shows how to visualize a streaming point cloud sequence by using a Point Cloud Viewer block.

Featured Examples

Track Marker Using Simulink Images

Track a marker on a whiteboard by using images of the Simulink.ImageType data type.

Open Script

Multicore Simulation of Video Processing System

Run a video processing system on multiple cores using dataflow execution domain in Simulink®.

Open Model

Color-based Road Tracking

Use color information to detect and track road edges set in primarily residential settings where lane markings may not be present. The Color-based Tracking example illustrates how to use the Color Space Conversion block, the Hough Transform block, and the Kalman Filter block to detect and track information using hue and saturation.

Open Model

Lane Departure Warning System

Detect and track road lane markers in a video sequence and notifies the driver if they are moving across a lane. The example illustrates how to use the Hough Transform, Hough Lines and Kalman Filter blocks to create a line detection and tracking algorithm. The example implements this algorithm using the following steps: 1) Detect lane markers in the current video frame. 2) Match the current lane markers with those detected in the previous video frame. 3) Find the left and right lane markers. 4) Issue a warning message if the vehicle moves across either of the lane markers.

Open Model

Tracking Cars Using Foreground Detection

Detect and count cars in a video sequence using Gaussian mixture models (GMMs).

Open Model

Tracking Based on Color

Track a person's face and hand using a color-based segmentation method.

Open Model

Abandoned Object Detection

Track objects at a train station and to determine which ones remain stationary. Abandoned objects in public areas concern authorities since they might pose a security risk. Algorithms, such as the one used in this example, can be used to assist security officers monitoring live surveillance video by directing their attention to a potential area of interest.

Open Model

Tracking Cars Using Optical Flow

Detect and track cars in a video sequence using optical flow estimation.

Open Model

Concentricity Inspection

Inspect the concentricity of both the core and the cladding in a cross-section of optical fiber. Concentricity is a measure of how centered the core is within the cladding.

Open Model

Motion Detection

Use sum of absolute differences (SAD) method for detecting motion in a video sequence. This example applies SAD independently to four quadrants of a video sequence. If motion is detected in a quadrant, the example highlights the quadrant in red.

Open Model

Pattern Matching

Use the 2-D normalized cross-correlation for pattern matching and target tracking.

Open Model

Object Counting

Use basic morphological operators to extract information from a video stream. In this case, the model counts the number of staples in each video frame. Note that the focus and lighting change in each video frame.

Open Model

Scene Change Detection

Segment video in time. The algorithm in this example can be used to detect major changes in video streams, such as when a commercial begins and ends. It can be useful when editing video or when you want to skip ahead through certain content.

Open Model

Surveillance Recording

Process surveillance video to select frames that contain motion. Security concerns mandate continuous monitoring of important locations using video cameras. To efficiently record, review, and archive this massive amount of data, you can either reduce the video frame size or reduce the total number of video frames you record. This example illustrates the latter approach. In it, motion in the camera's field of view triggers the capture of "interesting" video frames.

Open Model

Traffic Warning Sign Recognition

Recognize traffic warning signs, such as Stop, Do Not Enter, and Yield, in a color video sequence.

Open Model

Video Stabilization

Remove the effect of camera motion from a video stream.

Open Script

Barcode Recognition Using Live Video Acquisition

Use the From Video Device block provided by Image Acquisition Toolbox™ to acquire live image data from a Point Grey Flea® 2 camera into Simulink®. The example uses the Computer Vision Toolbox™ to create an image processing system which can recognize and interpret a GTIN-13 barcode. The GTIN-13 barcode, formally known as EAN-13, is an international barcode standard. It is a superset of the widely used UPC standard.

Open Model

Video Mosaicking

Create a mosaic from a video sequence. Video mosaicking is the process of stitching video frames together to form a comprehensive view of the scene. The resulting mosaic image is a compact representation of the video data. The Video Mosaicking block is often used in video compression and surveillance applications.

Open Model

Edge Detection Using Live Video Acquisition

Use the From Video Device block provided by Image Acquisition Toolbox™ to acquire live image data from a Hamamatsu C8484 camera into Simulink®. The Prewitt method is applied to find the edges of objects in the input video stream.

Open Model