Applying SHAP on a Reinforcement Learning Algorithm

2023 6 5
2 个回答
更新时间：2025 3 5
14 次查看（30 天）

0 个投票

I am trying to Apply SHAP on a reinforcement Learning Algorithm and I am not sure if MATLAB has the required SHAP packages such as shap.DeepExplainer() which is a python package.
If anyone has any further information on how to apply SHAP on the Neural Network agent of a Reinforcement Learning model, please let me know.

6 个评论
显示 4更早的评论 隐藏 4更早的评论

Juliana Tavora
Juliana Tavora 2023-11-23
I have been wondering the same @Mahsa Raeisinezhad. Have you found further info? Documentation on shapley is not too clear and never mentions SHAP
Mahsa Raeisinezhad
Mahsa Raeisinezhad 2023-11-25
编辑：Mahsa Raeisinezhad 2023-11-25
figure;
for p=1:d
scatter(shap(:,sortedPredictorIndicesMEAN(p)), ... % x-value of each point is the shapley value
p*ones(n,1), ... % y-value of each point is an integer corresponding to a predictor (to be jittered below)
[], ... % Marker size for each data point, taking the default here
normalize(table2array(tbl(1:n,sortedPredictorIndicesMEAN(p))),'range',[1 256]), ... % Colors based on feature values
'filled', ... % Fills the circles representing data points
'YJitter','density', ... % YJitter according to the density of the points in this row
'YJitterWidth',0.8)
if (p==1)
hold on;
end
end
title('Shapley Summary plot');
xlabel('Shapley Value (impact on model output)')
yticks([1:d]);
yticklabels(tbl.Properties.VariableNames(sortedPredictorIndicesMEAN));
% Set colormap as desired
colormap(CoolBlueToWarmRedColormap); % This colormap is like the one used in many Shapley summary plots
% colormap(parula); % This is the default colormap
cb= colorbar('Ticks', [1 256], 'TickLabels', {'Low', 'High'});
cb.Label.String = "Scaled Feature Value";
cb.Label.FontSize = 12;
cb.Label.Rotation = 270;
set(gca, 'YGrid', 'on');
xline(0, 'LineWidth', 1);
hold off;
%%
function colormap = CoolBlueToWarmRedColormap()
% Define start point, middle luminance, and end point in L*ch colorspace
% https://www.mathworks.com/help/images/device-independent-color-spaces.html
% The three components of L*ch are Luminance, chroma, and hue.
blue_lch = [54 70 4.6588]; % Starting blue point
l_mid = 40; % luminance of the midpoint
red_lch = [54 90 6.6378909]; % Ending red point
nsteps = 256;
% Build matrix of L*ch colors that is nsteps x 3 in size
% Luminance changes linearly from start to middle, and middle to end.
% Chroma and hue change linearly from start to end.
lch=[[linspace(blue_lch(1), l_mid, nsteps/2), linspace(l_mid, red_lch(1), nsteps/2)]', ... luminance column
[linspace(blue_lch(2), red_lch(2), nsteps)]', ... chroma column
[linspace(blue_lch(3), red_lch(3), nsteps)]']; ... hue column
% Convert L*ch to L*a*b, where a = c * cos(h) and b = c * sin(h)
lab=[lch(:,1) lch(:,2).*cos(lch(:,3)) lch(:,2).*sin(lch(:,3))];
% Convert L*a*b to RGB
colormap=lab2rgb(lab,'OutputType','uint8');
Mahsa Raeisinezhad
Mahsa Raeisinezhad 2023-11-25
myAct = @(env) predict_01(pretrainedAgent, env);
% number of itereations
shap=zeros(n,d);
figure;
hold on;
for i = 1:n
explainer = shapley(myAct, env);
explainer = fit(explainer, env);
shap(i,:)=explainer.ShapleyValues{:,2};
plot(explainer)
explainer_{i} = explainer;
end
Mahsa Raeisinezhad
Mahsa Raeisinezhad 2023-11-25
function myAct_ = predict_01(tbl, env)
obsInfo = getObservationInfo(env);
actInfo = getActionInfo(env);
agent = rlPPOAgent(obsInfo,actInfo);
actor = getActor(agent);
actorNet = getModel(actor);
% Get the i-th slice of transposedArray
dataArray = table2array(tbl);
dlObservationsF = dlarray(dataArray, 'C');
% dlObservationsF = dlarray(dataArray(i,:), 'C');
% Predict using the actor network
myAct = predict(actorNet, dlObservationsF);
myAct_ = extractdata(myAct);
myAct_ = max(myAct);
end
Mahsa Raeisinezhad
Mahsa Raeisinezhad 2023-11-25
The above is how tried to use SHAP in Matlab, I created a function handle of my Neural Network (agent) predictions using the Environment and applied SHAP on each decision, but I still highly recommend transfering to Python and using Python shap packages.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（2 个）

Mahsa Raeisinezhad
Mahsa Raeisinezhad 2023-11-23

0 个投票

I decided to transfer everything in Python and use python packages. I used ONNX and Tensorflow for transferring everything. Hopefully if I have time in the future I write my own code to create same outcomes in Matlab.
Ive J
Ive J 2023-11-23

0 个投票

Document and refs are clear enough if you're aware enough what your intentions are. Follow this example if you're interested to still stick to MATLAB.

类别

在 帮助中心 和 File Exchange 中查找有关 Pretrained Networks from External Platforms 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by