Since the background is neutral colored, I'd probably convert to hsv color space
hsv = rgb2hsv(rgbImage);
and then threshold the S channel
background = hsv(:,:,2) < 0.2; % Or whatever value works.
After that it depends on what your mathematical definition of "remove" means. Do you mean to make it pure black? Do you want to crop for some reason? What do you need to do?
See my File Exchange for full blown demos on hsv color segmentation: http://www.mathworks.com/matlabcentral/fileexchange/?term=authorid%3A31862
