transolverNetwork

R2026b

Create Transolver neural network

Since R2026b

collapse all in page

    Syntax

    net = transolverNetwork(inputSize,outputSize)
    net = transolverNetwork(inputSize,outputSize,Name=Value)

    Description

    net = transolverNetwork(inputSize,outputSize) creates a Transolver [1] neural network with the specified input channel size and output channel size. The Transolver neural network architecture uses a physics-attention mechanism to learn spatial representations of physical systems. This architecture works well for solving partial differential equations (PDEs) and for other scientific computing tasks.

    example

    net = transolverNetwork(inputSize,outputSize,Name=Value) specifies additional options using one or more name-value arguments. For example, NumBlocks=8 creates a Transolver neural network with eight Transolver blocks.

    example

    Examples

    collapse all

    Create a Transolver neural network with an input channel size of 3, which corresponds to three input features, and an output channel size of 2.

    Create the network.

    net = transolverNetwork(3,2)
    net =
  dlnetwork with properties:

         Layers: [9x1 nnet.cnn.layer.Layer]
    Connections: [12x2 table]
     Learnables: [94x3 table]
          State: [0x3 table]
     InputNames: {'input'}
    OutputNames: {'conv1d_2'}
    Initialized: 1

    View a summary of the neural network architecture.

    summary(net)
       Initialized: true

   Number of learnables: 2.9M

   Inputs:
      1   'input'   1x3xNaN input with format "SCB"

    Create a Transolver neural network with more blocks, a smaller hidden size, and dropout regularization.

    For this example, create a Transolver neural network with five input channels and three output channels. Specify a smaller hidden size than the default of 256, specify more blocks than the default 4, and enable dropout. View a summary of the neural network architecture.

    net = transolverNetwork(5,3, ...
    NumBlocks=6, ...
    HiddenSize=128, ...
    NumHeads=4, ...
    DropoutProbability=0.1);
summary(net)
       Initialized: true

   Number of learnables: 1.1M

   Inputs:
      1   'input'   1x5xNaN input with format "SCB"

    Input Arguments

    collapse all

    Number of input channels, specified as a positive integer.

    The number of input channels must match the number of input variables at each spatial point. For example, if the input data has two variables per point (such as the x-velocity and y-velocity of a fluid), set inputSize to 2.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Number of output channels, specified as a positive integer.

    The number of output channels matches the number of output variables that the neural network predicts at each spatial point. For example, to predict one variable (such as the temperature at each point), set outputSize to 1.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Name-Value Arguments

    collapse all

    Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

    Example: net = transolverNetwork(inputSize,outputSize,NumBlocks=8) creates a Transolver neural network with eight Transolver blocks.

    Number of Transolver slices used in the physics-attention mechanism, specified as a positive integer.

    The physics-attention mechanism learns to decompose the input domain into spatial regions called slices. More slices allow the model to capture finer-grained spatial structures.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Number of hidden channels, specified as a positive integer.

    The Transolver neural network maps the input channels to a latent space with HiddenSize channels. The Transolver blocks operate in this latent space, and a final projection layer converts the latent representation back to the output channel size. Increasing the hidden size increases the capacity of the neural network, but also increases the number of learnable parameters.

    The value of HiddenSize must be divisible by NumHeads.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Activation function used in the feedforward sublayers of the Transolver blocks, specified as one of these values:

    • "gelu" — Gaussian error linear unit (GELU) activation. The function adds geluLayer objects to the output network.

    • "relu" — Rectified linear unit (ReLU) activation. The function adds reluLayer objects to the output network.

    • "swish" — Swish activation. The function adds swishLayer objects to the output network.

    • "tanh" — Hyperbolic tangent activation. The function adds tanhLayer objects to the output network.

    • "sigmoid" — Sigmoid activation. The function adds sigmoidLayer objects to the output network.

    • "leakyrelu" — Leaky rectified linear unit (Leaky ReLU) activation. The function adds leakyReluLayer objects to the output network.

    • "prelu" — Parametric rectified linear unit (PReLU) activation. The function adds preluLayer objects to the output network.

    • "elu" — Exponential linear unit (ELU) activation. The function adds eluLayer objects to the output network.

    • "softplus" — Softplus activation. The function adds softplusLayer objects to the output network.

    • "identity" — Identity activation. This operation is equivalent to having no activation function. The function adds identityLayer objects to the output network.

    Normalization layer type used in the Transolver blocks, specified as one of these values:

    Number of Transolver blocks in the neural network, specified as a positive integer.

    More blocks add depth and capacity to the model.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Number of attention heads in the physics-attention mechanism, specified as a positive integer.

    The value of NumHeads must evenly divide HiddenSize.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Dropout probability for the Transolver blocks, specified as a scalar in the range [0,1).

    The neural network applies dropout after the physics-attention sublayer and after the feedforward sublayer in each Transolver block. A value of 0 means no dropout.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Padding token value, specified as a numeric scalar.

    The neural network uses this value to identify padded elements in variable-length input sequences. The neural network masks positions where all input channels equal the padding token. Masked elements do not affect predictions for non-padded elements. This behavior enables batched inference on sequences of different spatial lengths.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Output Arguments

    collapse all

    Transolver neural network, returned as an initialized dlnetwork object.

    The neural network accepts input data with the format "SCB" (spatial, channel, batch) or "SC" (spatial, channel). The spatial dimension corresponds to the number of spatial points (for example, mesh nodes), and the channel dimension corresponds to inputSize. The network output has the same spatial dimension as the input, and its channel dimension is equal to outputSize.

    References

    [1] Wu, Haixu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. "Transolver: A Fast Transformer Solver for PDEs on General Geometries." arXiv, February 4, 2024. https://doi.org/10.48550/arXiv.2402.02366.

    Version History

    Introduced in R2026b

    See Also

    | | | | | | |

    Topics