Complete transformer model (Encoder + Decoder + Interconections)

2024 8 2
1 个回答
更新时间：2024 10 5
10 次查看（30 天）

0 个投票

Hello
I am wondering if there is already a Matlab keyboard warrior that has coded (on Matlab) a full transformer model:
  1. Inputs: Input Embedding + Positional Encoding
  2. Encoder: Multihead Attention + Add & Normalisation + Feedforward + Add & Normatisation
  3. Outputs: Output Embedding + Positional Encoding
  4. Decoder: Masked Multihead Attention + Add & Normalisation + Multihead Attention + Add & Normalisation + Feedforward + Add & Normatisation
  5. Final: Linear and Softmax.
Including all the interconnections between them.
Thank you
Will

请先登录，再回答此问题。

回答（1 个）

Yash Sharma
Yash Sharma 2024-8-5

0 个投票

Hi Will,
You can take a look at the following file exchange submission.
Transformers Models:- https://www.mathworks.com/matlabcentral/fileexchange/107375-transformer-models

2 个评论
显示 无 隐藏 无

WIll Serrano
WIll Serrano 2024-8-7
Hello Yash
Thank you for your answer.
I read that one, it is based on a pre-trained transformer and it does not directly represent the transformer components. As well it provides the same functionality as a normal LSTM for text classification.
It is acknowledged transformers with attention are somehow superior to Deep Learning based on LSTM, however, I have yet to prove it myself.
Thank you
Will
WIll Serrano
WIll Serrano 2024-10-5
As it seems nobody has answered, I have cracked the code myself.

请先登录，再进行评论。

类别

在 帮助中心 和 File Exchange 中查找有关 Specialized Power Systems 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by