Color-based Tracker


Given an image of a person, we would like to know which pixels correspond to the person's skin, and which are background or clothes. This can be useful in tracking, for example, the position of a person's head or hands. In this example, we select a training sample of skin, to which we fit a gaussian distribution. We use this distribution to predict if other pixels correspond to skin. We also try to predict if pixels belong to the background.

We try several different representations of color. First, we use standard RGB, which has a lot of error in classification. Then we convert to HSV, which provides significantly better results. We also attempt to "put HSV on a circle". This is to deal with the discontinuity of Hue around the extremities, which are supposed to represent the same value (Hue of 1.0 == Hue of 0.0). So we consider Hue as angle, and Saturation as radius, and compute X, Y coordinates on the circle, which replace H and S. V is left as is. There is very little difference with this method, but it is expected that we would see a larger difference if the skin hue values were clustered around the extremes.

Finally, we compute P'skin = max(Pskin - Pbackground, 0), which removes a large number of background pixels.

You can dowload a complete video that shows a side-by-side display with the (output.mpg). The source code for this project is freely available.

Copyright © 2003 Christopher DeCoro