Multiclass Classification using SVM (tuning C parameter)

Question

Assem Ibrahim 2023-12-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2060524-multiclass-classification-using-svm-tuning-c-parameter

回答： Shubham 2023-12-28

I'm trying to create a multiclass classifier using SVM with a linear kernel and to tune the hyper parameter "C" from 2^-10 to 2^10 to get the best model, I've tried to use a for loop for doing that that pass through the 1024 value of "C" and get the best accuracy, but it takes too long time. I've heard that there is a a parameter called "OptimizeHyperparameters" that can do so. I'm asking if that is right and the way to use it is ok or not. THANKS!

%Create Classifier
t = templateLinear('Solver','sgd');
vars = [optimizableVariable('BoxConstraint', [2^-10, 2^10], 'Type', 'real', 'Transform', 'log')];
opts = struct('Optimizer', 'bayesopt', 'ShowPlots', true);
%Train a linear classifier based on the transformed data and the correct classification labels 
Cmdl = fitcecoc(X_train{3},y_train{3}, 'OptimizeHyperparameters', vars, 'HyperparameterOptimizationOptions', opts);
 
%% Access results
bestHyperparameters = Cmdl.HyperparameterOptimizationResults.XAtMinObjective;
bestModel = Cmdl.HyperparameterOptimizationResults.bestPoint;
% Display best BoxConstraint value
bestBoxConstraint = bestHyperparameters.BoxConstraint;
disp(['Best BoxConstraint: ' num2str(bestBoxConstraint)]);
%% Evaluate Classifier
predicted_labels = predict(Cmdl,X_test{3});
C = confusionmat(y_test{3}, predicted_labels);
accuracy_c = sum(diag(C)) / sum(C(:)) 

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shubham 2023-12-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2060524-multiclass-classification-using-svm-tuning-c-parameter#answer_1379056

在 MATLAB Online 中打开

Hi Assem,

In MATLAB, you can use the OptimizeHyperparameters option within fitcecoc to automatically tune hyperparameters such as the BoxConstraint (C) for SVM. Your approach using Bayesian optimization ('bayesopt') is a good choice, as it can be more efficient than a simple grid search or a random search because it uses past evaluation results to choose the next values to evaluate.

Here's a corrected and optimized version of your code:

% Create a template for a linear SVM
t = templateLinear('Solver','sgd');
% Define the range for the BoxConstraint (C) hyperparameter
vars = optimizableVariable('BoxConstraint', [2^-10, 2^10], 'Type', 'real', 'Transform', 'log');
% Set the options for hyperparameter optimization
opts = struct('Optimizer', 'bayesopt', 'ShowPlots', true, 'MaxObjectiveEvaluations', 30);
% Train a multiclass model using ECOC (Error-Correcting Output Codes) with the linear SVM template
% and optimize the BoxConstraint hyperparameter
Cmdl = fitcecoc(X_train{3}, y_train{3}, 'Learners', t, 'OptimizeHyperparameters', vars, ...
    'HyperparameterOptimizationOptions', opts);
% Access results
bestHyperparameters = Cmdl.HyperparameterOptimizationResults.XAtMinObjective;
bestModel = Cmdl.HyperparameterOptimizationResults.bestPoint;
% Display best BoxConstraint value
bestBoxConstraint = bestHyperparameters.BoxConstraint;
disp(['Best BoxConstraint: ', num2str(bestBoxConstraint)]);
% Evaluate Classifier
predicted_labels = predict(Cmdl,X_test{3});
C = confusionmat(y_test{3}, predicted_labels);
accuracy_c = sum(diag(C)) / sum(C(:));
% Display the accuracy
disp(['Accuracy: ', num2str(accuracy_c)]);

Note the following changes and suggestions:

The Learners option is properly set to the template t you created.
MaxObjectiveEvaluations is set to 30 in opts to limit the number of evaluations and speed up the optimization process. You can adjust this number based on your computational budget and time constraints.
The bestModel variable is not necessary as XAtMinObjective already provides the best hyperparameters found.
The accuracy is also displayed at the end of the script.

By using Bayesian optimization with fitcecoc, MATLAB will intelligently search the hyperparameter space for the best BoxConstraint value, which should be faster and potentially more effective than exhaustively searching through all 1024 values. Remember that you can adjust the MaxObjectiveEvaluations to perform more or fewer evaluations depending on your time constraints and computational resources.