Hyper-parameter optimization

3 次查看（过去 30 天）
Elena Casiraghi
Elena Casiraghi 2019-9-21
回答： Elena Casiraghi 2019-9-24
1) When training an ECOC classifier for multiclass classification, with knn as base learner, how can I change the minimized function (from the classification error to a loss function I want to define)?
I'm now using this code (where the loss function is in the last line of code). If Preds are the predicted classes, labels are the true classes, N is the numebr of sample, my loss is:
myLoss = double(sum(abs(resPreds - labels)))/double(N); % this is the loss function I wish to minimize
% variable labels contains the labels of training data
tknn = templateKNN('Distance', @distKNN); % I WOULD LIKE TO USE THIS DISTANCE
N = size(XKnn,1);
c = cvpartition(N,'LeaveOut');
% Use leave one out
mdlknnCecoc = fitcecoc(XKnn,labelsRed, ...
'OptimizeHyperparameters','auto', ...
'HyperparameterOptimizationOptions',struct( 'UseParallel',...
true,'CVPartition',c), 'Learners',tknn);
resPreds = predict(mdlknnCecoc, XKnn); % I don't know why kfoldPredict function does not work
myLoss = double(sum(abs(resPreds - labels)))/double(N); % this is the loss function I wish to minimize
  2 个评论
Don Mathis
Don Mathis 2019-9-23
Is it important for you to use ECOC for this? fitcknn directly supports multiclass problems.
Elena Casiraghi
Elena Casiraghi 2019-9-24
ECOC classifiers provide better results (at least in my first experiments).
Anyhow I solved the problem.
I defined a function myLoss:
function loss = myLoss(C,S,W,Cost)
% C is an n-by-K logical matrix with rows indicating the class to which the corresponding observation belongs.
% The column order corresponds to the class order in CVMdl.ClassNames.
% Construct C by setting C(p,q) = 1 if observation p is in class q, for each row.
% Set every element of row p to 0.
%
% S is an n-by-K numeric matrix of negated loss values for the classes.
% Each row corresponds to an observation.
% The column order corresponds to the class order in CVMdl.ClassNames.
% The input S resembles the output argument NegLoss of kfoldPredict.
%
% W is an n-by-1 numeric vector of observation weights.
% If you pass W, the software normalizes its elements to sum to 1.
%
% Cost is a K-by-K numeric matrix of misclassification costs.
% For example,
% Cost = ones(K) – eye(K) specifies a cost of 0 for correct classification and 1 for misclassification.
if size(C,2)>1; [~,Y] = max(C,[],2);
else; Y = C; end
if size(S,2)>1; [~,Yfit] = max(S,[],2);
else; Yfit = S; end
cost = diag(Cost(Y,Yfit));
loss = sum(abs(Y-Yfit).*W.*cost);
end
and I'm now using this code, which works :) fine (To test it I optimized the ECOC doing an exhaustive search). However, I will follow your suggestion and I will also test knn
c = cvpartition(nTrain,'LeaveOut');
% train knn on the reduced training set
% non uso matrici di costo!
num = optimizableVariable('n',[1,maxK],'Type','integer');
cod = optimizableVariable('coding',{'onevsone','onevsall','ternarycomplete'},'Type','categorical');
fun = @(x)kfoldLoss(fitcecoc(X,Y,'CVPartition',c,'Coding', char(x.coding),...
'Learners',templateKNN('Distance', @distKNN,'NumNeighbors',x.n)),'LossFun',@myLoss);
bestParams = bayesopt(fun,[num,cod], 'Verbose',0);

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Elena Casiraghi
Elena Casiraghi 2019-9-24
ECOC classifiers provide better results (at least in my first experiments).
Anyhow I solved the problem.
I defined a function myLoss:
function loss = myLoss(C,S,W,Cost)
% C is an n-by-K logical matrix with rows indicating the class to which the corresponding observation belongs.
% The column order corresponds to the class order in CVMdl.ClassNames.
% Construct C by setting C(p,q) = 1 if observation p is in class q, for each row.
% Set every element of row p to 0.
%
% S is an n-by-K numeric matrix of negated loss values for the classes.
% Each row corresponds to an observation.
% The column order corresponds to the class order in CVMdl.ClassNames.
% The input S resembles the output argument NegLoss of kfoldPredict.
%
% W is an n-by-1 numeric vector of observation weights.
% If you pass W, the software normalizes its elements to sum to 1.
%
% Cost is a K-by-K numeric matrix of misclassification costs.
% For example,
% Cost = ones(K) – eye(K) specifies a cost of 0 for correct classification and 1 for misclassification.
if size(C,2)>1; [~,Y] = max(C,[],2);
else; Y = C; end
if size(S,2)>1; [~,Yfit] = max(S,[],2);
else; Yfit = S; end
cost = diag(Cost(Y,Yfit));
loss = sum(abs(Y-Yfit).*W.*cost);
end
and I'm now using this code, which works :) fine (To test it I optimized the ECOC doing an exhaustive search). However, I will follow your suggestion and I will also test knn
c = cvpartition(nTrain,'LeaveOut');
% train knn on the reduced training set
% non uso matrici di costo!
num = optimizableVariable('n',[1,maxK],'Type','integer');
cod = optimizableVariable('coding',{'onevsone','onevsall','ternarycomplete'},'Type','categorical');
fun = @(x)kfoldLoss(fitcecoc(X,Y,'CVPartition',c,'Coding', char(x.coding),...
'Learners',templateKNN('Distance', @distKNN,'NumNeighbors',x.n)),'LossFun',@myLoss);
bestParams = bayesopt(fun,[num,cod], 'Verbose',0);

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Discriminant Analysis 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by