Does DeepNetworkDesigner support Transformer Layers?

35 次查看（过去 30 天）
Subhadip
Subhadip 2024-1-14
回答： David Ho 2024-1-18
I want to add Transofmrer Layers while designing the network for deep learning using MATLAB DeepNetworkDesigner, but it doesn't show me such option. How can I add Transofmer Layers to the network designed using DeepNetworkDesigner?

请先登录，再回答此问题。

回答（2 个）

Matt J
Matt J 2024-1-14
If the transformer layer is a custom layer that you have created for yourself, you would import it to deepNetworkDesigner from the workspace, as explained here,
https://www.mathworks.com/help/deeplearning/ref/deepnetworkdesigner-app.html
David Ho
David Ho 2024-1-18
Hi Subhadip,
In R2023b, the following layers are supported in Deep Network Designer that are commonly used in transformers:
  • selfAttentionLayer
  • wordEmbeddingLayer (requires Text Analytics Toolbox)
  • sinusoidalPositionEncodingLayer
  • positionEmbeddingLayer
  • embeddingConcatenationLayer
  • indexing1dLayer
  • patchEmbeddingLayer (requires Computer Vision Toolbox)
For more information you can refer to the list of deep learning layers page and the links in it.
Best regards,
David

类别

在 Help Center 和 File Exchange 中查找有关 Build Deep Neural Networks 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by