Semantic Segmentation

Semantic image segmentation

Semantic segmentation associates each pixel of an image with a class label, such as flower, person, road, sky, or car. Use the Image Labeler and the Video Labeler apps to interactively label pixels and export the label data for training a neural network.

Input image of a seascape, then a series of cubes representing a deep learning network and a semantic segmented output image of the input.

Apps

Image Labeler	Label images for computer vision applications
Video Labeler	Label video for computer vision applications

Functions

expand all

Load Training Data

`combine`	Combine data from multiple datastores
`countEachLabel`	Count occurrence of pixel or box labels
`groundTruth`	Ground truth label data
`imageDatastore`	Datastore for image data
`pixelLabelDatastore`	Datastore for pixel label data
`pixelLabelTrainingData`	Create training data for semantic segmentation from ground truth

Augment and Preprocess Training Data

`balancePixelLabels`	Balance pixel labels by oversampling block locations in large images (Since R2020a)
`imwarp`	Apply geometric transformation to image
`imcrop`	Crop image
`imresize`	Resize image
`transform`	Transform datastore
`randomAffine2d`	Create randomized 2-D affine transformation
`randomWindow2d`	Randomly select rectangular region in image (Since R2021a)
`centerCropWindow2d`	Create rectangular center cropping window

Design Semantic Segmentation Deep Learning Networks

`unet`	Create U-Net convolutional neural network for semantic segmentation (Since R2024a)
`unet3d`	Create 3-D U-Net convolutional neural network for semantic segmentation of volumetric images (Since R2024a)
`deeplabv3plus`	Create DeepLab v3+ convolutional neural network for semantic image segmentation (Since R2024a)
`focalCrossEntropy`	Compute focal cross-entropy loss (Since R2020b)

Segment Images Using Deep Learning

`semanticseg`	Semantic image segmentation using deep learning
`imsegsam`	Perform automatic full image segmentation using Segment Anything Model (SAM) (Since R2024b)
`segmentAnythingModel`	Pretrained Segment Anything Model (SAM) for semantic segmentation (Since R2024a)

Visualize Segmentation

`labeloverlay`	Overlay label matrix regions on 2-D image
`volshow`	Display volume
`insertObjectMask`	Insert masks in image or video stream (Since R2020b)

Evaluate Segmentation Results

`evaluateSemanticSegmentation`	Evaluate semantic segmentation data set against ground truth
`bfscore`	Contour matching score for image segmentation
`dice`	Sørensen-Dice similarity coefficient for image segmentation
`generalizedDice`	Generalized Sørensen-Dice similarity coefficient for image segmentation (Since R2021a)
`jaccard`	Jaccard similarity coefficient for image segmentation
`segmentationConfusionMatrix`	Confusion matrix of multi-class pixel-level image segmentation (Since R2020b)
`semanticSegmentationMetrics`	Semantic segmentation quality metrics

Topics

Get Started

Getting Started with Semantic Segmentation Using Deep Learning
Segment objects by class using deep learning.
Choose Function to Visualize Detected Objects
Compare visualization functions.

Create Ground Truth Data for Semantic Segmentation

Label Pixels for Semantic Segmentation
Label pixels for training a semantic segmentation network by using a labeling app.
How Labeler Apps Store Exported Pixel Labels
Learn how the labeling apps store pixel label data.

Create Training Data for Semantic Segmentation

Training Data for Object Detection and Semantic Segmentation
Create training data for object detection or semantic segmentation using the Image Labeler or Video Labeler.
Datastores for Deep Learning (Deep Learning Toolbox)
Learn how to use datastores in deep learning applications.
Get Started with Image Preprocessing and Augmentation for Deep Learning
Preprocess data for deep learning applications with deterministic operations such as resizing, or augment training data with randomized operations such as random cropping.

Featured Examples

New

Automatically Label Ground Truth Using Segment Anything Model

Produce pixel labels for semantic segmentation using the Segment Anything Model (SAM) in the Image Labeler app. The SAM is an automatic segmentation technique that you can use to segment object regions to label with just a few clicks, or automatically segment the entire image and instantaneously create labels for selected regions. In this example, you interactively label pixels for semantic segmentation in two ways.