mapanything

R2026b

Create pretrained MapAnything model for 3-D scene reconstruction from images

Since R2026b

expand all in page

    Description

    Add-On Required: This feature requires the Computer Vision Toolbox Model for MapAnything Network add-on.

    Use the mapanything object to create a pretrained MapAnything model for 3-D scene reconstruction from images. MapAnything is a feed-forward transformer model that combines visual understanding with geometric reasoning to reconstruct 3-D scenes from multi-view images in true metric scale. Unlike traditional structure-from-motion methods that require all the images used in a reconstruction to come from the same calibrated camera, the MapAnything model can process images from different cameras or heterogeneous sources. The model can operate on images alone, but can optionally leverage additional geometric inputs such as camera intrinsic parameters, camera poses, or depth maps to improve reconstruction accuracy. MapAnything processes image sequences automatically, handling large data sets through block-based processing with automatic alignment. You can use the model to generate camera poses, estimate camera intrinsic parameters, produce depth maps, and create dense 3-D point clouds without first performing manual feature extraction or traditional structure-from-motion preprocessing.

    Creation

    Syntax

    maObj = mapanything(imds)
    maObj = mapanything(imds,Name=Value)

    Description

    maObj = mapanything(imds) creates a pretrained MapAnything model object maObj from the images in the image datastore imds. To perform 3-D reconstruction of the images in the image datastore after creating the object, use the reconstruct object function.

    example

    maObj = mapanything(imds,Name=Value) sets writable properties using one or more name-value arguments. For example, IsPoseMetricScale=true specifies to use metric scale information from the input camera poses during reconstruction.

    Input Arguments

    expand all

    Images to use for 3-D reconstruction, specified as an imageDatastore object. The image collection must consist of grayscale or RGB images. The images can be unordered and do not need to be captured using the same camera. To get better reconstruction results, use images that have significant overlap (60—80%) between consecutive views.

    This argument sets the ImageSource property.

    Name-Value Arguments

    expand all

    Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

    Example: mapanything(imds,CameraPoses=poses,IsPoseMetricScale=true) specifies to use metric scale information from the input camera poses poses during reconstruction.

    Camera intrinsic parameters, specified as a cameraIntrinsics object or B-by-1 vector of cameraIntrinsics objects. Specify a single cameraIntrinsics object when all images have been captured using the same camera and share the same intrinsic parameters. When the images are captured from multiple cameras and have different intrinsic parameters, specify this argument as a B-by-1 vector of cameraIntrinsics objects, where B is the number of images in the image datastore imds. If images are captured from multiple camera sources and intrinsics are known only for a subset of those sources, you must specify the corresponding cameraIntrinsics objects for the known cameras. For the remaining sources, you must specify an empty cameraIntrinsics object.

    This argument sets the Intrinsics property.

    Camera poses of the input images, specified as a B-by-1 vector of rigidtform3D objects, where B is the number of images in the image datastore imds.

    This argument sets the CameraPoses property.

    Depth map of the input images, specified as an imageDatastore or a B-by-1 cell array of images, where B is the number of images in the image datastore imds. Each depth map must have the same dimensions as its associated input image. You must replace any invalid depth values (NaN or Inf) with 0.

    This argument sets the DepthMaps property.

    Option to use metric scale information from input depth maps, specified as a logical 0 (false) or 1 (true). To preserve absolute scale in reconstruction, when the depth maps specified by the DepthMaps argument are in metric scale, specify this argument as true.

    This argument sets the IsDepthMetricScale property.

    Data Types: logical

    Option to use metric scale information from input camera poses, specified as a logical 0 (false) or 1 (true). To preserve absolute scale in reconstruction, when the camera poses specified in the CameraPoses argument are in metric scale, specify this argument as true.

    This argument sets the IsPoseMetricScale property.

    Data Types: logical

    Properties

    expand all

    This property is read-only after object creation. To set this property, use the imds argument during object creation.

    Input images to use for 3-D reconstruction, specified as an imageDatastore object.

    This property is read-only after object creation. To set this property, use the Intrinsics argument during object creation.

    Camera intrinsic parameters, specified as a cameraIntrinsics object or B-by-1 vector of cameraIntrinsics objects.

    This property is read-only after object creation. To set this property, use the CameraPoses argument during object creation.

    Camera poses of the input images, specified as a B-by-1 vector of rigidtform3D objects, where B is the number of images in the image datastore imds.

    This property is read-only after object creation. To set this property, use the DepthMaps argument during object creation.

    Depth map of the input images, specified as an imageDatastore or a B-by-1 cell array of images, where B is the number of images in the image datastore imds.

    This property is read-only after object creation. To set this property, use the IsDepthMetricScale argument during object creation.

    Option to use metric scale information from input depth maps, specified as a logical 0 (false) or 1 (true).

    This property is read-only after object creation. To set this property, use the IsPoseMetricScale argument during object creation.

    Option to use metric scale information from input camera poses, specified as a logical 0 (false) or 1 (true).

    Object Functions

    reconstructReconstruct 3-D scene using MapAnything model
    releaseGPUMemoryRelease GPU memory allocated to MapAnything model

    Examples

    collapse all

    This example uses:

    Open Live Script

    Download and extract the tum_rgbd_data.zip file. This file includes an images subfolder that contains a sequence of indoor images from the TUM RGB-D data set [1]. 

    if ~exist("tum_rgbd_data.zip","file")
    websave("tum_rgbd_data.zip","https://ssd.mathworks.com/supportfiles/3DReconstruction/tum_rgbd_data.zip");
    unzip(fullfile("tum_rgbd_data.zip"), pwd);
end

    Create an image datastore by specifying the path to the folder containing the image files, and subsample every 5th frame to reduce processing time when you perform 3-D reconstruction.

    imds = imageDatastore("sfmTrainingDataTUMRGBD\images");
numFiles = numel(imds.Files);
imds = subset(imds,1:5:numFiles);

    Load the camera intrinsics and camera poses. Subsample the poses using the same indices as the images so that the number of poses matches the number of images in the datastore.

    camInfo = load(fullfile("sfmTrainingDataTUMRGBD","cameraInfo.mat"));
camIntrinsics = camInfo.intrinsics;
camPoses = camInfo.cameraPoses;
camPoses = transpose(camPoses(1:5:numFiles));

    Create a mapanything object using the image datastore, camera intrinsics, and camera poses, and reconstruct the 3-D scene from the images.

    maObj = mapanything(imds,Intrinsics=camIntrinsics,CameraPoses=camPoses);
maOutputs = reconstruct(maObj);
    Processing Individual blocks
---------------------------------------------
Processing Block 1/1

Processing point clouds
---------------------------------------------
Processing point cloud 21/21

    Extract the generated point cloud. Then, visualize the point cloud by using the pcviewer (Point Cloud Toolbox) function, and set the vertical axis and vertical axis direction to match the coordinate system of the TUM RGB-D data set. Modify the view of the point cloud visualization to focus on a region of interest by specifying the low-level camera properties of the axes object.

    pcArray = readall(maOutputs.PointClouds);
pcMerged = pccat(pcArray);
ax = pcviewer(pcMerged);

    ax.CameraPosition = [-1.5185 -6.5178 -11.9106];
ax.CameraUpVector = [0.0520 -0.8878 0.4572];
ax.CameraViewAngle = 15;

    If you are using a GPU resource to run the model, you can release the GPU memory allocated to the model after processing to enable additional models to run on the GPU.

    releaseGPUMemory(maObj);

    References

    [1] Sturm, Jürgen, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. “A Benchmark for the Evaluation of RGB-D SLAM Systems.” 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2012, 573–80. https://doi.org/10.1109/IROS.2012.6385773.

    References

    [1] Keetha, Nikhil, Norman Müller, Johannes Schönberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, et al. “MapAnything: Universal Feed-Forward Metric 3D Reconstruction.” Preprint, arXiv, 2025. https://doi.org/10.48550/arXiv.2509.13414.

    Version History

    Introduced in R2026b

    See Also

    Objects

    Functions