generateFromImage

R2026b

Generate 3D asset from one or more images

Since R2026b

collapse all in page

Syntax

meshObj = generateFromImage(trellisObj,images)
meshObj = generateFromImage(trellisObj,images,mask)
meshObj = generateFromImage(___,Name=Value)
[meshObj,info] = generateFromImage(___)

Description

Add-On Required: This feature requires the 3D Asset Processing Library for MATLAB add-on.

meshObj = generateFromImage(trellisObj,images) generates a 3D asset from the image or images specified by images, using the trellis object trellisObj, and returns the generated 3D asset as a Mesh object, meshObj.

example

meshObj = generateFromImage(trellisObj,images,mask) generates a 3D asset from the region of the image or images images defined by the foreground mask or masks mask. Use this syntax when you want to specify the foreground region instead of relying on automatic background removal.

meshObj = generateFromImage(___,Name=Value) specifies options using one or more name-value arguments in addition to any combination of input arguments from previous syntaxes. For example, TextureSize=2048 sets the resolution of the generated texture image to 2048 pixels.

[meshObj,info] = generateFromImage(___) returns additional information, such as the preprocessed image or images after background removal.

Note

This feature requires Deep Learning Toolbox™ and a GPU with at least 16 GB of memory and a CUDA® compute capability of 7.0 or higher.

Examples

collapse all

This example uses:

Open Live Script

Note: This example requires Deep Learning Toolbox and a GPU with at least 16 GB of memory and a CUDA compute capability of 7.0 or higher.

Create a trellis object.

trellisObj = trellis;

Read a single RGB image into the workspace.

img = imread("ObjaverseAssetImage.jpeg");
imshow(img)

Generate a 3D asset from the image by using the generateFromImage object function of the trellis object.

meshObj = generateFromImage(trellisObj,img);

Display the generated 3D asset.

show(meshObj)

Generate a 3D asset from the image with custom generation parameters. Return the preprocessing information info, enabling you to inspect the image after background removal.

[meshObj_custom,info] = generateFromImage(trellisObj,img,GeometryGenerationGuidanceStrength=8.5,GeometryGenerationSamplingSteps=50,...
    TextureGenerationGuidanceStrength=3.5,TextureGenerationSamplingSteps=50,SimplifyRatio=0.9,TextureSize=2048,Seed=42);

Display the 3D asset generated using the custom parameters.

show(meshObj_custom)

Display the preprocessed image to inspect the background removal applied before asset generation.

figure
imshow(info.PreprocessedImage)
title("Preprocessed Image")

Input Arguments

collapse all

Trellis 3D asset generator, specified as a trellis object.

Input image or images, specified as one of these options:

  • M-by-N-by-3 numeric array — Single RGB image for single-view generation.

  • M-by-N-by-3-by-B numeric array — Set of B RGB images for multi-view generation, where B is the number of images of the same object from different views or angles.

  • ImageDatastore object — Each use of the read object function of the datastore must return one M-by-N-by-3 RGB image. The function reads all images from the datastore, treats them as multi-view images of a single object, and returns a single generated 3D asset.

  • B-element cell array — Each cell contains one M-by-N-by-3-by-B RGB image of the same object from a different view. The function treats all images in the cell array as multi-view input, and returns a single generated 3D asset.

For single-view input, the model infers nonvisible regions of the object from learned priors.

Foreground mask or masks that identify the object region in the image. The default value is [], which enables automatic background removal. Specify the mask as one of these options:

  • [] — No mask. The function automatically removes the background.

  • M-by-N logical matrix — Single foreground mask for single-view input.

  • M-by-N-by-B logical array — Stack of B masks for multi-view input, where each mask corresponds to one image.

  • B-element cell array — Each cell contains one M-by-N-by-3-by-B logical mask.

Name-Value Arguments

collapse all

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: generateFromImage(trellisObj,images,TextureSize=2048,Seed=42) sets the texture resolution to 2048 pixels and fixes the generation seed to 42.

Guidance strength for geometry generation, specified as a numeric scalar in the range [0, 10]. Use higher values to produce a 3D shape that more closely matches the input image. Use lower values to enable the model to infer more of the asset from its training data, such as when the image shows only part of the object.

Number of sampling steps for geometry generation, specified as an integer in the range [1, 50]. Higher values produce a more refined and stable 3D shape but increase generation time.

When you use multiple images as input, specify this argument as a value greater than the number of input images.

Guidance strength for texture generation, specified as a numeric scalar in the range [0, 10]. Use higher values when you want the generated texture to closely match the colors and surface details of the input image. If the output shows noise or visual artifacts, reduce this value to enable the model to rely more on its training data for a potentially cleaner result.

Number of sampling steps for texture generation, specified as an integer in the range [1, 50]. Higher values produce more refined and stable textures but increase generation time.

When you use multiple images as input, specify this argument as a value greater than the number of input images.

Mesh simplification ratio, specified as a numeric scalar in the range [0, 1]. Use higher values to produce a lighter mesh that is faster to render and easier to work with downstream. Use lower values to retain more geometric detail in the final mesh. For example, a value of 0.9 removes approximately 90% of the vertices while preserving the overall shape of the asset.

Size of the generated texture image, specified as an integer in the range [512, 2048]. Use higher values to increase the sharpness and detail of textures.

Random seed for generation, specified as a nonnegative integer. By default, the function uses a different random seed each time, which produces a different result for every call. Specify a fixed seed value to ensure repeatability each time you run the function with the same inputs and settings.

Display progress bar during object creation, specified as a logical 1 (true) or 0 (false). By default, the trellis function displays a progress bar while it initializes the object and sets up the model.

Output Arguments

collapse all

Generated 3D asset, returned as a Mesh object.

Additional output information, returned as a structure with this field:

  • PreprocessedImage — Image after background removal, returned as an M-by-N-by-3 numeric array.

Limitations

macOS does not support generating 3D mesh assets from images or text prompts.

Version History

Introduced in R2026b

See Also

| | | | | |

Topics