How to create a positional encoder layer using deep network designer?
8 次查看(过去 30 天)
显示 更早的评论
Im creating the network architecture of my transformer but I don't know how to put the positional information "positional encoding" with Deep Network Designer, this is a code that I have so far...
function lgraph = setNeuralNetworkArchitecture(inputSize, numClasses)
% Create layer graph
lgraph = layerGraph();
% Add layer branches
tempLayers = [
sequenceInputLayer([13 24 8],"Name","sequence")
% Añadir positional encoding
lambdaLayer(@(x) addPositionalEncoding(x, encodingDim), 'Name', 'positionalEncoding')
flattenLayer("Name","flatten")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = selfAttentionLayer(16,64,"Name","selfattention");
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
additionLayer(2,"Name","addition")
fullyConnectedLayer(6,"Name","fc_1")
softmaxLayer("Name","softmax")
classificationLayer("Name","classoutput")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"positionalEncoding","flatten");
lgraph = connectLayers(lgraph,"flatten","selfattention");
lgraph = connectLayers(lgraph,"flatten","addition/in2");
lgraph = connectLayers(lgraph,"selfattention","addition/in1");
end
0 个评论
回答(1 个)
Siraj
2023-8-24
Hii! @Luis Gabriel Macias Santillan It is my understanding that you want to add a positional encoder layer using the deep network designer app available in MATLAB. According to my research, the deep network designer doesn’t have a positional encoder layer that could be directly added to the network.
You can refer to the following documentation to understand how we can integrate a custom deep-learning layer.
Also, you can check out the following file, which has an implementation for a positional encoder.
Hope this helps.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!