Main Content

medicalSegmentAnythingModel

Pretrained Medical Segment Anything Model (MedSAM) for medical image segmentation

Since R2024b

expand all in page

    Description

    The Medical Segment Anything Model (MedSAM) is a Segment Anything Model (SAM) fine-tuned for medical image segmentation on a large-scale medical image data set. Use the medicalSegmentAnythingModel object and its object functions to interactively segment objects in 2-D medical images using visual prompts. A medicalSegmentAnythingModel object configures MedSAM for semantic segmentation of objects in a medical image without retraining the model. To segment an image, you must first use the extractEmbeddings object function to extract the image embeddings from the MedSAM image encoder. Then, use the segmentObjectsFromEmbeddings object function to segment objects from the image embeddings using the mask decoder.

    Note

    This functionality requires Deep Learning Toolbox™, Computer Vision Toolbox™, and the Medical Imaging Toolbox™ Model for Medical Segment Anything Model. You can install the Medical Imaging Toolbox Model for Medical Segment Anything Model from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    Creation

    Syntax

    medsam = medicalSegmentAnythingModel

    Description

    medsam = medicalSegmentAnythingModel loads a pretrained Medical Segment Anything Model. To use this model to interactively segment objects in medical images using visual prompts, specify it to the extractEmbeddings object function to extract image embeddings, and then use the segmentObjectsFromEmbeddings object function on the embeddings along with a visual prompt.

    example

    Object Functions

    extractEmbeddingsExtract image embeddings from Medical Segment Anything Model (MedSAM) encoder
    segmentObjectsFromEmbeddingsSegment objects in medical image using Medical Segment Anything Model (MedSAM) image embeddings

    Examples

    collapse all

    This example uses:

    Open Live Script

    Create a Medical Segment Anything Model (MedSAM) object.

    medsam = medicalSegmentAnythingModel;

    Load an X-ray image into the workspace. Visualize the image.

    medImg = medicalImage("forearmXrayImage1.dcm");
img = medImg.Pixels;
figure
imshow(img,[])

    Extract the image embeddings from the X-ray image using the extractEmbeddings object function.

    embeddings = extractEmbeddings(medsam,img);

    Prepare the image display for MedSAM interactive segmentation. Create a new figure and axes, and display the image.

    f = figure;
ax = axes(f);
imshow(img,[],Parent=ax);

    Draw an ROI that contains an object to segment as a Rectangle object on the image by using the drawrectangle function.

    roi = drawrectangle(ax);

    To segment the object, run the MedSAM decoder on the image embeddings using the segmentObjectsFromEmbeddings object function. Specify the bounding box visual prompt BoundingBox as the position of the rectangle ROI.

    mask = segmentObjectsFromEmbeddings(medsam,embeddings,size(img),BoundingBox=roi.Position);
overlayedImg = insertObjectMask(rescale(img),mask);

    Visualize the segmentation mask overlaid on the image in the image display.

    figure
imshow(overlayedImg)

    References

    [1] Ma, Jun, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. “Segment Anything in Medical Images.” Nature Communications 15, no. 1 (January 22, 2024): 654. https://doi.org/10.1038/s41467-024-44824-z.

    Version History

    Introduced in R2024b

    See Also

    Topics

    External Websites