Detection and tracking of human face and hands are curial for gesture recognition. In this paper, a skin segmentation framework is presented, where a robust elliptical model in HSV color space is developed, which promises not only to better model the skin color, but also to significantly lower the complexity of the hand detection process. In our approach, hand regions are identified and detected in Hue Saturation Value (HSV) color space that is proven to correspond closer to human perception of hand color than the traditional RGB color space. The obtained results show the potential of the method proposed and it yields good performance and has robustness with respect to illumination as well as with respect to image noise.