Box Constraint SVM - Mistake?

9 次查看（过去 30 天）
Ismail Kuzu
Ismail Kuzu 2017-11-14
编辑： Lukasz Szyc 2020-8-7
Hi,
in the documantary for the Support Vector Regression the box constraint is described as a parameter penalizing vectors laying outside the epsilon-margin. A higher box constraint causes higher costs. www.mathworks.com/help/stats/understanding-support-vector-machine-regression.html
In the documantary of the regression-learner-app it is quite different:
"The box constraint controls the penalty imposed on observations with large residuals. A larger box constraint gives a more flexible model. A smaller value gives a more rigid model, less sensitive to overfitting." www.mathworks.com/help/stats/choose-regression-model-options.html
Should it not be that a smaller box constraint leads to a more flexible model and reduces the risk of overfitting? Did I got it wrong?

请先登录，再回答此问题。

回答（2 个）

Lukasz Szyc
Lukasz Szyc 2018-4-17
编辑：Lukasz Szyc 2020-8-7
Increasing the boxconstraint leads (or at least can lead) to fewer support vectors as described in the documentation. It might seem counterintuitive, but the number of support vectors is not a particularly good measure of model complexity. With a small boxconstraint you allow more points in the margin. An inf large box constraint you allow no data points in the margin. The smaller boxconstraint the wider the margin, and usually more data points lie within the margin. All the data points in the margin are always support vectors. In other words: a large boxconstraint = misclassification heavily weighted, smaller margin, fewer supporting vectors, sometimes also more complex model, higher overfitting risk.
Ahmad Obeid
Ahmad Obeid 2019-2-6
编辑：Ahmad Obeid 2019-2-6
I think that in the standard definition of the cost function of SVMs, the penalyzing parameter, AKA the regularization parameter, multiplies the first part of the cost function (as opposed to to say linear regression, where the second part of the cost function is multiplied). This may be the source of your confusion, as you might have thought that both definitions are the same. For better illustration, see the two equations in the attached image. I obtained these equations from the machine learning course on Coursera, taught by Dr. Andrew.
Hope it helps.

类别

在 Help Center 和 File Exchange 中查找有关 Gaussian Process Regression 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by