Dear friends, can I ask how to add an attention mechanism layer to TD3's critic network?

21 次查看（过去 30 天）
SJ
SJ 2024-3-11
回答： Lokesh 2024-10-28，20:14
nObs=7；
% Define the attention layer
attentionSize=nObs;
attentionLayer = attentionLayer(attentionSize, 'AttentionLayer');
% Main path
mainPath = [
featureInputLayer(nObs,'Name',"StateInLyr")
attentionLayer
fullyConnectedLayer(128)
fullyConnectedLayer(128)
concatenationLayer(1,2,'Name',"concat")
reluLayer
fullyConnectedLayer(128)
reluLayer
fullyConnectedLayer(1,'Name',"CriticOutLyr")
];
% Action path
actionPath = [
featureInputLayer(nAct,'Name',"ActionInLyr")
fullyConnectedLayer(128,'Name',"fc2")
];
% Convert to layergraph object and connect layers
criticNet = layerGraph(mainPath);
criticNet = addLayers(criticNet, actionPath);
criticNet = connectLayers(criticNet,"fc2","concat/in2");
critic1 = rlQValueFunction(criticNet,obsInfo,actInfo,...
'ObservationInputNames',"StateInLyr",'ActionInputNames',"ActionInLyr");
critic2 = rlQValueFunction(criticNet,obsInfo,actInfo,...
'ObservationInputNames',"StateInLyr",'ActionInputNames',"ActionInLyr");
%%%%%%%%%%%%%%%%%%%%%%%%%
The code runs with the following error：
Error using string
Conversion from element failed. Element 1 must be convertible to a string scalar.
error: deep.internal.sdk.dag2dlnetwork_legacy>iCanApplyFix (line 221 )
layersWithIssues = string(analyzer.Issues.Layers);
error: deep.internal.sdk.dag2dlnetwork_legacy (line 69 )
canApplyFix = iCanApplyFix(analyzer);
error: rl.internal.model.createInternalModel (line 12 )
model = deep.internal.sdk.dag2dlnetwork_legacy(model);
error: rlQValueFunction (line 122 )
model = rl.internal.model.createInternalModel(model, nameValueArgs.UseDevice, inputSize, outputSize);
error: cfkz (line 114)
critic1 = rlQValueFunction(criticNet,obsInfo,actInfo,...

请先登录，再回答此问题。

回答（1 个）

Lokesh
Lokesh 2024-10-28，20:14
Hello SJ,
It seems that you are encountering an issue with your code due to an unconnected input to the 'attentionLayer'.
I faced a similar error when running the setup in MATLAB R2022b. You can observe this issue by using the following command:
analyzeNetwork(criticNet)
Using the 'analyzeNetwork' command can help you visualize the network structure and identify any disconnected layers or inputs, making it easier to debug and configure your network correctly.
To resolve this error, note that the 'attentionLayer' requires two inputs: 'hidden' and 'encoder' as per R2022b. Currently, only one input is connected, which is causing the problem. Ensure that both inputs are properly connected to the 'attentionLayer'.
Hope this helps!

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by