Objective function for Lasso

2 次查看（过去 30 天）
Alex
Alex 2012-6-20
Hi,
We would like to compute the Bayesian Information Criterion (BIC) for some models estimated using the lasso function in the statistics toolbox. We have been trying to work out exactly what the objective function is that the lasso function is optimising. We have used the following formulation, based on what we can glean from doc lasso:
% generate random dependent variable and design matrix
Y = rand(50,1);
P = rand(50,100);
L = .5; % arbitrary lambda value
A = 1; % set alpha to 1
% Fit lasso model for arbitrary lambda
[Beta Info] = lasso(P, Y, 'Lambda', L, 'Alpha', A);
Yh = P*Beta; % compute predicted scores
R = Y-Yh-Info.Intercept; % compute residual term based on formulation in help lasso
SR = sum(R)./(2*length(Y));
K = size(Be,1);
n = length(Y);
F = SR+L*(((1-A)/2)*sum(Beta.^2)+(A.*sum(abs(Beta)))); % objective function for elastic net as per help lasso
BIC = -2.*log(F)+(K*(log(n))); % compute BIC
One thing that seems a little odd to us is that SR is computed using actual rather than squared residuals. This means that positive and negative values can cancel each other out. The fomrmula in help lasso does not indicate that squared residuals are used, so we were wondering whether the above interpretation is correct.
Thanks and regards,
Alex

请先登录，再回答此问题。

回答（1 个）

Ilya
Ilya 2012-6-20
You are asking for a definition of BIC for lasso. This is not really a MATLAB question. You might have better luck posting it to sci.stat.math or a similar group.
I am not an expert. For what it's worth, using residuals instead of residuals squared looks odd to me too. (But I don't see a problem with lasso documentation; it's a problem with your definition of BIC.) Two other things look odd to me as well: Including the lasso penalty in the variance term and using the number of all predictors in the model as the number of degrees of freedom. A quick Google search finds this paper:
http://www.stanford.edu/~hastie/Papers/dflasso.pdf
Their definition of BIC for lasso looks nothing like yours.

类别

在 Help Center 和 File Exchange 中查找有关 Linear Regression 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by