Complete transformer model (Encoder + Decoder + Interconections)

Question

WIll Serrano 2024-8-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2142456-complete-transformer-model-encoder-decoder-interconections

评论： WIll Serrano 2024-8-7

Hello

I am wondering if there is already a Matlab keyboard warrior that has coded (on Matlab) a full transformer model:

Inputs: Input Embedding + Positional Encoding
Encoder: Multihead Attention + Add & Normalisation + Feedforward + Add & Normatisation
Outputs: Output Embedding + Positional Encoding
Decoder: Masked Multihead Attention + Add & Normalisation + Multihead Attention + Add & Normalisation + Feedforward + Add & Normatisation
Final: Linear and Softmax.

Including all the interconnections between them.

Thank you

Will

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Yash Sharma 2024-8-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2142456-complete-transformer-model-encoder-decoder-interconections#answer_1494601

Hi Will,

You can take a look at the following file exchange submission.

Transformers Models:- https://www.mathworks.com/matlabcentral/fileexchange/107375-transformer-models

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

WIll Serrano 2024-8-7

Hello Yash

Thank you for your answer.

I read that one, it is based on a pre-trained transformer and it does not directly represent the transformer components. As well it provides the same functionality as a normal LSTM for text classification.

It is acknowledged transformers with attention are somehow superior to Deep Learning based on LSTM, however, I have yet to prove it myself.

Thank you

Will

请先登录，再进行评论。