Detect and Track Face

Introduction

Object detection and tracking are important in many computer vision applications, including activity recognition, automotive safety, and surveillance. In this example, you design a system in Simulink® to detect a face in a video frame, identify the facial features and track these features. The output video frame contains the detected face and the features tracked. If a face is not visible or goes out of focus, the system tries to re-acquire the face and then perform the tracking. This example is designed to detect and track a single face.

Example Model

close
open_system('DetectAndTrackFace');

Setup

This example uses the From Multimedia File block to read the video frames from the video file. The Detection and Tracking subsystem takes in a video frame and provides a bounding box for the face and feature points within the bounding box as its output to the Draw Annotations subsystem. This subsystem inserts in the image a rectangle for the bounding box and markers for the feature points.

Detection and Tracking

In this example, the vision.CascadeObjectDetector System object™ detects the location of the face in the current video frame. The cascade object detector uses the Viola-Jones detection algorithm and a trained classification model for detection. After the face is detected, facial feature points are identified using the "Good Features to Track" method proposed by Shi and Tomasi.

Then the vision.PointTracker System object™ tracks the identified feature points by using the Kanade-Lucas-Tomasi (KLT) feature-tracking algorithm. For each point in the previous frame, the point tracker attempts to find the corresponding point in the current frame. Then the estimateGeometricTransform function estimates the translation, rotation, and scale between the old points and the new points. This transformation is applied to the bounding box around the face.

Although it is possible to use the cascade object detector on every frame, it is computationally expensive. This technique can also fail to detect the face, such as when the subject turns or tilts his head. This limitation comes from the type of trained classification model used for detection. In this example, you detect the face once, and then the KLT algorithm tracks the face across the video frames. The detection is performed again only when the face is no longer visible or when the tracker cannot find enough feature points.

The ability to perform Dynamic memory allocation in MATLAB functions (Simulink) allows the usage of the previously mentioned System objects and methods inside the MATLAB® Function block.

open_system('DetectAndTrackFace/Detection and Tracking')

Draw Annotations

The bounding box corner points and feature point locations are used to draw on the output video frame. The Draw Shapes block draws the bounding box. The feature points are drawn using the Draw Markers block.

open_system('DetectAndTrackFace/Draw Annotations')

Results

The following display shows the detected face with the feature points.

The following display shows the tracked face and feature points.

References

Viola, Paul A., and Michael J. Jones. "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE CVPR, 2001.

Lucas, Bruce D., and Takeo Kanade. "An Iterative Image Registration Technique with an Application to Stereo Vision." International Joint Conference on Artificial Intelligence, 1981.

Lucas, Bruce D., and Takeo Kanade. "Detection and Tracking of Point Features." Carnegie Mellon University Technical Report CMU-CS-91-132, 1991.

Shi, Jianbo, and Carlo Tomasi. "Good Features to Track." IEEE Conference on Computer Vision and Pattern Recognition, 1994.

ZKalal, Zdenek, Krystian Mikolajczyk, and Jiri Matas. "Forward-Backward Error: Automatic Detection of Tracking Failures." International Conference on Pattern Recognition, 2010