extractEmbeddings

Extract image embeddings from Medical Segment Anything Model (MedSAM) encoder

Since R2024b

collapse all in page

    Syntax

    embeddings = extractEmbeddings(medsam,I)

    Description

    Add-On Required: This feature requires the Medical Imaging Toolbox Model for Medical Segment Anything Model add-on.

    embeddings = extractEmbeddings(medsam,I) extracts the embeddings of the input medical image I from the encoder of a Medical Segment Anything Model (MedSAM), medsam, by running a forward pass on the image encoder.

    Note

    This functionality also requires Deep Learning Toolbox™ and Computer Vision Toolbox™ along with the Medical Imaging Toolbox™ Model for Medical Segment Anything Model add-on.

    example

    Examples

    collapse all

    This example uses:

    Open Live Script

    Create a Medical Segment Anything Model (MedSAM) object.

    medsam = medicalSegmentAnythingModel;

    Load a heart ultrasound image into the workspace. Visualize the image.

    img = imread("heartUltrasoundImage.png"); 
figure
imshow(img,[])

    Extract the image embeddings from the heart ultrasound image.

    embeddings = extractEmbeddings(medsam,img);

    Visualize some of the embeddings, which capture the features of the image.

    figure
montage(embeddings(:,:,1:6),DisplayRange=[])

    Input Arguments

    collapse all

    Medical Segment Anything Model, specified as a medicalSegmentAnythingModel object.

    Image or batch of images from which to extract embeddings, specified as a 2-D numeric matrix, 3-D numeric array, or 4-D numeric array depending on the type and number of images.

    Type of ImageNumber of ImagesData Format
    Grayscale imageSingle 2-D image2-D numeric matrix of size H-by-W
    Grayscale imageBatch of B 2-D images4-D numeric array of size H-by-W-by-1-by-B
    RGB imageSingle 2-D image3-D numeric array of size H-by-W-by-3
    RGB imageBatch of B 2-D images4-D numeric array of size H-by-W-by-3-by-B

    H and W are the height and width, respectively, of the input image I. To run the segmentation on a GPU, specify the input image I explicitly as a gpuArray (Parallel Computing Toolbox).

    Data Types: single | double | int8 | int16 | int32 | uint8 | uint16 | uint32

    Output Arguments

    collapse all

    Image embeddings extracted from the Medical Segment Anything Model encoder, returned as a 64-by-64-by-256 or 64-by-64-by-256-by-B numeric array, depending on the type and number of input images.

    Number of ImagesEmbeddings Format
    Single grayscale or RGB image64-by-64-by-256 array
    Batch of B grayscale or RGB images64-by-64-by-256-by-B array

    The image embeddings are representations of corresponding input images that encode the spatial information, features, and context of the input images.

    Extended Capabilities

    expand all

    Version History

    Introduced in R2024b

    See Also

    |

    Topics