File Exchange

image thumbnail

Feature selection with SVM-RFE

version 1.3.0.0 (5.42 KB) by

Support vector machine recursive feature elimination (SVM-RFE), with correlation bias reduction

33 Downloads

Updated

View License

SVM-RFE is a powerful feature selection algorithm in bioinformatics. It is a good choice to avoid overfitting when the number of features is high.
However, it may be biased when there are highly correlated features. We propose a "correlation bias reduction" strategy to handle it. See our paper (Yan et al., Feature selection and analysis on correlated gas sensor data with recursive feature elimination", 2015).
This file is an implementation of both our method and the original SVM-RFE, including the linear and RBF kernel. **LibSVM is needed**
Thanks to the SVM-KM and spider toolbox!

Comments and Ratings (20)

Mingyu Lee

Can I apply the non-linear version to a SVM with a polynomial kernel or is it just for RBF kernels?

Thank you.

jie wang

Shiang Qi

Hi Ke Yan,I intend to use SVM-RFECBR with RBF kernel to do feature selection, then use linear SVM to test the testing data based on the features that are selected, am I right? Or the two steps(feature selection and testing ) should use the same kernel???
Look forward to your reply, thank you.

elahe afshari

Hi,
W in svm-rfe :
w=sigma(alpha(k)*x(k)*y(k))

In to program
w = fs_model.SVs' * fs_model.sv_coef;

Is Formula One and two are equal?????????

sigma(alpha(k)*x(k)*y(k))== fs_model.SVs' * fs_model.sv_coef;
??????????

Ke Yan

Ke Yan (view profile)

Hi Hadeel, I don't know how you edited the code, but you may try to find this line:
type = sprintf('-s 0 -t %d -c %f -g %f -q',kerOpt.type,kerOpt.C,kerOpt.g);
in the sub-function trainSVM, and change it into:
type = sprintf('-s 0 -t %d -c %f -g %f -q -v %d',kerOpt.type,kerOpt.C,kerOpt.g, cross_val);
where cross_val is your number of cross validation folds.

Hi Ke Yan how we can use cross validation in svmtrain to ensure there is no over fitting in our data . I try to use -v but I got this error
Struct contents reference from a non-struct array object.
Thank you for your efforts

Sir, I have a feature vector of size 100*33738. while using SVM+RFE+CBR feature selection method I' getting following error:
Requested 33738x33738 (0.5GB) array exceeds maximum array size preference.
Creation of arrays greater than this limit may take a long time and cause
MATLAB to become unresponsive. See array size limit or preference panel for
more information.

Error in cov (line 156)
c = (xc' * xc) ./ denom;
Please help me to solve this problem.

Giorgio

Giorgio (view profile)

For SVM-RFE source code: please have a look at this toolbox: http://www.mathworks.com/matlabcentral/fileexchange/56937-feature-selection-library

I ma currently a novice, can someone explain me how to run this code in matlab on a dataset?

Ke Yan

Ke Yan (view profile)

mahmoud elgohary, the output FTRANK has the most important features at the beginning

lower rank or higher rank is most important?

Trudie

Trudie (view profile)

Ke Yan

Ke Yan (view profile)

SAPTARSHI CHATTERJEE, the input of the function should be of the size nSamples-by-nFeatures. The output 'ftScore' is the score of the features in each iteration. It is not the formal output of the function. The output 'ftRank' should be used as the importance order of the features.

Sir after running the code for the feature vector of length 126*689 I'm getting ftScore as 64*689 matrix.how can I solve this problem?please help me.

Sir length of my input feature vector is 124*689.After applying SVM-RFE with CBR code my feature vector length become 63*689.can you please explain the reason...

Thanks a lot. I'm using Matlab 2015a and download libsvm-3.21.but unable to install it. can u please help me in this regard. after running mex -stup command following comments are coming:
MEX configured to use 'lcc-win32' for C language compilation.
Warning: The MATLAB C and Fortran API has changed to support MATLAB
variables with more than 2^32-1 elements. In the near future
you will be required to update your code to utilize the
new API. You can find more information about this at:
http://www.mathworks.com/help/matlab/matlab_external/upgrading-mex-files-to-use-64-bit-api.html.

To choose a different language, select one from the following:
mex -setup C++
mex -setup FORTRAN
by choosing C++ following errors are coming:
Error using mex
No supported compiler or SDK was found. For options, visit
http://www.mathworks.com/support/compilers/R2015a/win32.html.
can u please help me to solve this problem.

Ke Yan

Ke Yan (view profile)

SAPTARSHI CHATTERJEE, seems you are using the svmtrain function from Matlab itself. You should install libsvm and add it to path to replace Matlab's builtin SVM in order to use this svm-rfe function.

I'm facing some problem to run this code.the size of my feature vector is 126*689.using matlab15 when I'm trying to execute this code following errors are coming
Error using internal.stats.parseArgs (line 42)
Wrong number of arguments.

Error in svmtrain (line 292)
[kfun,optimMethod, plotflag, polyOrder, mlpParams, boxC, rbf_sigma, ...

Error in ftSel_SVMRFECBR>trainSVM (line 180)
model = svmtrain(X,Y,type);

Error in ftSel_SVMRFECBR (line 71)
[supVec,alpha_signed] = trainSVM(ft(:,ftIn),label,kerOpt);

Error in FRT_SEL (line 14)
[ftRank,ftScore] = ftSel_SVMRFECBR(FT,label,PARAM);
can u please help me solve this problem.

Updates

1.3.0.0

1. remove "sv_indices" in function trainSVM older versions of libSVM don't have it
2. add a simple support for multi-class problems

1.2.0.0

fixed a bug: changed
if isempty(model) || model.nSV == 0
to
if isempty(model) || sum(model.nSV) == 0

1.1.0.0

revise description

MATLAB Release Compatibility
Created with R2010a
Compatible with any release
Platform Compatibility
Windows macOS Linux

MATLAB Online Live Editor Challenge

Win cash prizes and have your live script featured on our website

Learn more

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video