LASSO with additional non-linear constraint

2 次查看（过去 30 天）
Anton Kovac
Anton Kovac 2017-2-13
评论： Alan Weiss 2017-2-21
Hi everyone,
I am really stacked.
I try to find optimal beta coefficients using LASSO technique, but with additional constraints. The problem looks like this.
subject to
Basically, I want to compute beta coefficients using lasso with constraint to be less than or equal to their sum of absolute value differences between them and other coefficients (because there are absolute values in non-linear constraint, I redefined constraints to appropriate form).
beta_b -> I want to compute
beta_a -> I previously computed (there should be parameters for the constraint function).
x -> numeric vector (n x 1)
X -> numeric matrix (n x m)
The lower index L means lagged, we need not consider it.
I tried to use fmincon solver, but I am stacked how should it look like.
I have this objective function
%%Objective function
function f = objfun(x, Y, M, lamb)
f = ((x*M - Y)^2 + lamb*abs(x));
end
And nonlinear constraints can look like this
function [c, ceq] = nonlcon(x, eps, p, betas_ref)
%%Inequality constraints
c(1) = x - betas_ref - (eps / p);
c(2) = betas_ref - x - (eps / p);
%%Equality constraints
ceq = [];
end
But the objective function need to be LASSO, so I do not know how can I put LASSO into this problem.
Please, can anybody help me with this problem?

请先登录，再回答此问题。

回答（1 个）

Alan Weiss
Alan Weiss 2017-2-13
For fmincon, or indeed any Optimization Toolbox™ solver, you need to put all of your control variables in one vector, typically called x. I cannot tell from your formulation which are your control variables. You say that you want to compute the beta_b, but I do not see beta_b in your equations. I cannot tell if your x and X are data or are control variables.
In any case, once you figure out which are your control variables, put them all in one vector. Then you can extract them in your objective function, something like this. If one control variable is y = an n-by-m matrix, and another is b = a length t vector, then
x = [y(:);b(:)]
and x is a nm + t length vector. Inside your objective function you would write something like
function fun = myobj(x,n,m)
y = x(1:n*m);
b = x(n*m+1:end);
y = reshape(y,n,m);
% more commands here
Then you would pass
fun = @(x)myobj(x,n,m)
as the objective function, using a method for passing extra parameters, the parameters n and m in this case, but you probably want to pass more parameters than that.
All that said, I don't know how well fmincon would do at finding an optimum for Lasso. You see, Lasso is an L^1 minimization, which is not smooth.
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
  2 个评论
Anton Kovac
Anton Kovac 2017-2-21
Thank you for your reply.
I tried some options but I did not solve it.
Honestly, it is not clear to me what is my control variables (I have not good background in math). I try to explain
I want to compute betas_b (the regression coefficients of LASSO), so the output of my optimization function should be vector. These betas_b are constrained to the constraint I wrote above.
I have computed betas_a (size(betas_a) = size(betas_b)), and I have the data x_i (vector to be regressed) and X_i (matrix of the regressors). Let's say I have additional parameter lambda (penalty coefficient of the lasso).
So, I do not know how can I build my objective function, if
  • I want the output to be a vector (not scalar as it is said in documentation) - but from the documentation I assume, that the output of the fmincon is size of x0.
  • I want LASSO type objective function
Further, I tried this
lamb = 0.001;
x = [X_i(:); x_i(:)]; % control variables??, X_i matrix, x_i vector
n = size(X_i, 1); % rows
m = size(X_i, 2); % columns
% Parameters for nonlinear constraints
eps = 0.05;
p = size(betas_ref.coefs{i}, 1); % scalar
b_ref = betas_ref.coefs{i}; % vector -> betas_a
fun = @(x)objfun(x, n, m);
mycon = @(x)nonlcon(x, eps, p, b_ref);
[x, fval] = fmincon(fun, x0, [], [], [], [], [], [], ...
mycon);
My objective function
%%Objective function
function fun = objfun(x, n, m)
M = x(1:n*m);
Y = x(n*m+1:end)';
M = reshape(M, n, m);
fun = lasso(M , Y);
end
And constraints
function [c, ceq] = nonlcon(x, eps, p, betas_ref)
% betas_ref -> vector of size(x)
% eps , p -> scalars
%%Inequality constraints
c(1) = x - betas_ref - (eps / p);
c(2) = betas_ref - x - (eps / p);
%%Equality constraints
ceq = [];
end
But I got an error
Index exceeds matrix dimensions.
Error in objfun (line 4) M = x(1:n*m);
But when I calculate it by hand it fits.
I think my way is not right. So could you please help me with these questions?
  • Can I just put to the objective function buit-in function (like lasso)? Something like
%%Objective function
function y = myobj(x)
%%code here
y = lasso(X, Y)
end
  • If not, how should look my objective function, when it should be the lasso?
  • Is my constraints definition right?
Finally, all I want to do is the equation (4) in following article (page 2)
http://www-bcf.usc.edu/~liu32/icdm2012.pdf
Alan Weiss
Alan Weiss 2017-2-21
You have so many misconceptions that it is hard to know where to begin.
You say you want your objective function to be a vector. You cannot have a vector-valued objective function in a minimization. Think about it, what should be minimized, the first component of the vector, the second, what? If you want to minimize the norm of the objective function vector, then you want to minimize that scalar value, so that is what you should specify as the objective function. In other words, if you want to minimize a vector-valued objective function, then you have not thought enough about what your problem is.
For another topic: control variables are the variables that can be adjusted to try to find an optimum. If you don't know which variables are the control variables, then you have no business trying to write an objective function. You have to figure that out first, before asking for any more help. As another way to think about it, you want to find some values that optimize something. Those values are your control variables, and the something is your objective function.
When you have figured out your control variables and scalar objective function, we can discuss things further.
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Support Vector Machine Regression 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by