I am trying to using `fminunc` top obtain the optimal theta in logistic regression, however I keep getting that:
fminunc stopped because it cannot decrease the objective function
along the current search direction.
Searching online, I found that this is usually the result of a gradient error which I am implementing in `logistic_costFunction.m`. I re-checked my work but I cannot spot the root cause.
I am not sure how to solve this issue, any help would be appreciated.
Here is my code
clear all; close all; clc;
x1 = linspace(0,3,50);
mqtrue = 5;
cqtrue = 30;
dat1 = mqtrue*x1+5*randn(1,50);
x2 = linspace(7,10,50);
dat2 = mqtrue*x2 + (cqtrue + 5*randn(1,50));
x = [x1 x2]';
subplot(2,2,1);
dat = [dat1 dat2]';
scatter(x1, dat1); hold on;
scatter(x2, dat2, '*'); hold on;
classdata = (dat>40);
[m, n] = size(x);
x = [ones(m, 1) x];
initial_theta = zeros(n + 1, 1);
[cost, grad] = logistic_costFunction(initial_theta, x, dat);
fprintf('Cost at initial theta (zeros): %f\n', cost);
fprintf('Gradient at initial theta (zeros): \n');
fprintf(' %f \n', grad);
options = optimset('GradObj', 'on', 'MaxIter', 400);
[theta, cost] = ...
fminunc(@(t)(logistic_costFunction(t, x, dat)), initial_theta, options);
logistic_costFunction.m
-----------------------
function [J, grad] = logistic_costFunction(theta, X, y)
m = length(y);
grad = zeros(size(theta));
H = sigmoid(X*theta);
T = y.*log(H) + (1 - y).*log(1 - H);
J = -1/m*sum(T);
for i = 1 : m
grad = grad + (H(i) - y(i)) * X(i,:)';
end
grad = 1/m*grad;
end
sigmoid.m
function g = sigmoid(z)
g = zeros(size(z));
g = 1 ./ (1 + (1 ./ exp(z)));
end
