Hi Raghu,
To optimize an ANFIS model using Bayesian optimization in MATLAB, you need to ensure that the optimization process is correctly set up, particularly the objective function that evaluates the model based on the hyperparameters. Your code has a few issues that need addressing to make it work properly. Here's a step-by-step guide to help you set up the optimization:
Key Steps:
- Data Preparation: Ensure your data is correctly split into training and testing sets.
- Objective Function: Define an objective function that trains the ANFIS model with given hyperparameters and returns a performance metric (e.g., mean squared error) on validation data.
- Bayesian Optimization: Use the bayesopt function to search for the best hyperparameters.
% Generate synthetic data
Daten = rand(100, 3);
Daten(:,3) = Daten(:,1) + Daten(:,2) + 0.1*randn(100, 1);
% Split into train and test
P = 0.7;
m = size(Daten, 1);
Training = Daten(1:round(P*m), :);
Testing = Daten(round(P*m)+1:end, :);
XTrain = Training(:, 1:2);
YTrain = Training(:, 3);
XTest = Testing(:, 1:2);
YTest = Testing(:, 3);
% Define cross-validation partition
cv = cvpartition(numel(YTrain), 'Holdout', 1/3);
% Define hyperparameters to optimize
vars = [
optimizableVariable('MF', [1, 20], 'Type', 'integer');
optimizableVariable('Epochs', [1, 100], 'Type', 'integer')
];
% Define the objective function
minfn = @(vars) kfoldLossFunction(XTrain, YTrain, cv, vars.MF, vars.Epochs);
% Optimize
results = bayesopt(minfn, vars, 'IsObjectiveDeterministic', false, ...
'AcquisitionFunctionName', 'expected-improvement-plus');
% Extract best hyperparameters
bestVars = bestPoint(results);
% Train final ANFIS model with best hyperparameters
opt = anfisOptions('InitialFIS', genfis1([XTrain YTrain], bestVars.MF), ...
'EpochNumber', bestVars.Epochs);
opt.DisplayErrorValues = 0;
opt.DisplayStepSize = 0;
fis = anfis([XTrain YTrain], opt);
% Evaluate ANFIS model
anfisOutput = evalfis(fis, XTest);
mseTest = mean((YTest - anfisOutput).^2);
fprintf('Test MSE: %.4f\n', mseTest);
% Objective function for cross-validation
function loss = kfoldLossFunction(XTrain, YTrain, cv, numMFs, numEpochs)
% Generate initial FIS
fis = genfis1([XTrain YTrain], numMFs);
% Initialize loss
loss = 0;
% Perform cross-validation
for i = 1:cv.NumTestSets
trainIdx = training(cv, i);
valIdx = test(cv, i);
% Train ANFIS
opt = anfisOptions('InitialFIS', fis, 'EpochNumber', numEpochs);
opt.DisplayErrorValues = 0;
opt.DisplayStepSize = 0;
trainedFis = anfis([XTrain(trainIdx, :) YTrain(trainIdx)], opt);
% Evaluate on validation set
valOutput = evalfis(trainedFis, XTrain(valIdx, :));
loss = loss + mean((YTrain(valIdx) - valOutput).^2);
end
% Average cross-validation loss
loss = loss / cv.NumTestSets;
end
Explanation:
- Data Preparation: Ensure that Daten is correctly split into training and testing datasets.
- Objective Function (kfoldLossFunction): This function trains the ANFIS model on the training data and evaluates it on the validation data. It returns the mean squared error (MSE) as the loss.
- Bayesian Optimization: Uses the bayesopt function to find the optimal number of membership functions (MF) and epochs (Epochs). The objective function is evaluated using k-fold cross-validation.
- Final Model Training: Once the best hyperparameters are found, train the ANFIS model on the entire training set and evaluate it on the test set.
