I've done something similar but with a green ball to make the image processing problem simpler. Here are some of the resources I used to get you started:
- Mouse Emulator: File Exchange submission that allows you to move the cursor and simulate clicks in MATLAB
- Image Acquisition Toolbox: Allows you to acquire video from a webcam
- Video and Image Processing Blockset: Provides algorithms for developing video processing applications.
- Color Segmentation Demo: Example of one approach for trying to detect the hand.
