Those would be discussed in the papers on the subject. They are listed here: http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics in this section:
21.4 Gesture and Pose Analysis
I imagine they talk about things like shadows, background clutter, speed of gesture changing, snapping "in between" unwanted gestures, bad camera angle, unrecognized gestures, etc.
