The person in my immediate family who had the reddest lips, was my father. The person who had the thinnest and whitest lips is my mother. And me... well I hope your program can figure out what to do with blue lips (poor circulation.)
You cannot use lip color to classify by genders.
If you were willing to surrender to stereotypes about gender and physical characteristics, then you would probably have higher accuracy looking at lip fullness than lip color.
