Hum!, the Hue channel (the one that contains the colour information) in a HSV image is the first one.
The limits of the blue hue depends on what you call blue, but it's in the region of 0.5 to 0.7.
So:
imghsv=rgb2hsv(img);
imgh=imghsv(:,:,2);
bluepixels = imgh >= 0.5 & imgh <= 0.7;
The saturation (2nd channel) is completely independent of the hue. You may want to filter pixels with very low saturation as they'll look almost black but it certainly won't help filtering out yellow (or green, or red, or any other colour)
