How do I find slack variables in SVM?/ Distance to the boundary?

Question

Mech Princess 2013-2-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/62757-how-do-i-find-slack-variables-in-svm-distance-to-the-boundary

评论： Aliza Rubenstein 2017-1-19

Hi, I used "svmtrain" to train the algorithm: svmStruct=svmtrain(xdata,group); I used "svmclassify" to classify.

My data is not perfectly linearly separable but I still used a linear classifier. In theory, allowances are made for by a slack variable. (Soft margin) I refer to the toolbox help where the theory is. It mentions the slack variable and 2 ways it is computed. http://www.mathworks.com/help/bioinfo/ug/support-vector-machines-svm.html

My issue is that, svmStruct does not save the slack variable. Neither can I find it in the function to recall it and save it.

If not, how can I find the distance from each data point to the boundary?

Can anyone help me with this? Thanks

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

the cyclist 2013-2-8

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/62757-how-do-i-find-slack-variables-in-svm-distance-to-the-boundary#answer_74316

It is the input parameter 'boxconstraint' to the svmtrain() command. The default value is 1.

5 个评论
显示 3更早的评论隐藏 3更早的评论

the cyclist 2013-2-8

Sorry! I thought the slack variable was the parameter C (which is actually the "penalty parameter").

I am not 100% sure that the slack variables have to be explicitly calculated to solve for the support vectors. (It's been a long time since I have used these techniques, and I actually don't have the toolbox at this time, to check.)

Sorry to have been more a distraction than a solution!

Mech Princess 2013-2-9

Thats fine. Hope someone will answer. Thanks

请先登录，再进行评论。

Answer 2

Ilya 2013-2-10

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/62757-how-do-i-find-slack-variables-in-svm-distance-to-the-boundary#answer_74504

By definition, a slack variable for observation x with label y (-1 or +1) is max(0,1-y*f), where f is the SVM prediction (soft score ranging from -inf to +inf). svmclassify does not return the scores, so you need to compute the SVM scores yourself. Start with the definition of the SVM model, compute kernel products, multiply by the alpha coefficients and add the bias term. It is easier than it sounds.

6 个评论
显示 4更早的评论隐藏 4更早的评论

Mech Princess 2013-2-11

编辑：Mech Princess 2013-2-11

在 MATLAB Online 中打开

Thank you. I did it with your suggestion. But now have another issue. See what I implemented in MATLAB below;

My data is positive integers<=6. sparse-mostly zeros.

n = no of support vectors for training data
p = no of features in the input data
y = label (+1/-1) (nx1 matrix)
x = data points (nxp matrix)
w = alpha*y*x; (px1)
N = no of testing data
X = testing data (Nxp matrix)
Y = label given by SVM (Nx1 vector)
b = bias (single value but made to a Nx1 vector)
s = slack (Nx1 vector)
Y(w*X-b) >= 1- s
s >= 1-Y(w*X-b); s<0 means no slack-no alpha

My w*X is very small (10^-14) So essentially (wX-b) = -b; s = 1+Yb

Now my slack variables give me only the label. Essentially, I have found another way to interpret the label using the already known results :(

The distance to the boundary is what I want. Do you have any suggestions what I am doing wrong? (I know this might not be a MATLAB problem anymore) Thanks again

Mark 2014-8-19

在 MATLAB Online 中打开

Thanks!

Only one remark: I think that this works fine if 'autoscale' is set to false (in the svmtrain function). If the data is scaled, you should also scale Xnew before you feed it to the kernel function:

shift = svm_struct.ScaleData.shift;
scale = svm_struct.ScaleData.scaleFactor;

Now you can scale Xnew:

XnewScaled = ( Xnew - shift ) .* scale;

and then use XnewScale in the kernel function as above:

f = kfun( sv, XnewScaled, kfunargs{:} )' * alphaHat + bias

Aliza Rubenstein 2017-1-19

Thanks for this information. I think it should be Xnew + shift, not Xnew - shift. shift is the negative of the mean.

请先登录，再进行评论。