This application demonstrates a fast, simple color tracking algorithm that
can be used to track faces, hands etc described in camshift.pdf.
The CAMSHIFT algorithm is a modification of the Meanshift algorithm which
is a robust statistical method of finding the mode (top) of a probability
distribution. Both CAMSHIFT and Meanshift algorithms exist in the library.
While it is a very fast and simple method of tracking, because CAMSHIFT
tracks the center and size of the probability distribution of an object,
it is only as good as the probability distribution that you produce for
the object. Typically the probability distribution is derived from
color via a histogram, although it could be produced from correlation,
recognition scores or bolstered by frame differencing or motion detection
schemes, or joint probabilities of different colors/motions etc.
In this application, we use only the most simplistic approach:
A 1-D Hue histogram is sampled from the object in an HSV color space version
of the image. To produce the probability image to track, histogram
"back projection" (we replace image pixels by their histogram hue value)
is used. The raw video image and backprojection is shown in the following
image:
Figure 1
Since we use such a simple method in this application, you must tune
the parameters for your camera and the lighting in the image as described
below.
Note that this filter requires that you have DirectShow 6.0 or greater installed.
DirectShow 6.0 for NT4.0, DirectShow 7.0 for Win 98 or Win2000).
You will also need a video camera source filter which should
be supplied by your USB camera vendor.
Microsoft* Windows* 2000 may be the most suitable operating system
for development since many vendors do not supply filters for
Windows NT 4.0. However, Windows 98 will be adequate for just
running the filter graph.
Controls
Figure 2
[1] Select input from USB video camera.
[2] Select input from "bouncing ball" simulation.
[3] Select input from an AVI movie file.
[4] Stop and start AVI movie file if [3]
is selected.
[5] Threshold the histogram bins to clean
noise.
[6] Select the number of histogram bins
to use in the 1-D Hue histogram.
[7] Minimum color saturation to accept
threshold on Hue pixels.
[8] Minimum brightness to accept threshold
on Hue pixels.
[9] Maximum brightness to accept threshold
on Hue pixels.
[10] View (top) Raw video; (middle) Histogram
backprojection; (bottom) Histogram bins.
[11] Select display window size -- on slower
machines, a smaller display window will run faster.
How to
Track a colored object (face, ball etc):
(a) Reset thresholds: Set Threshold [5] to zero; Saturation
[7] far left; Minimum brightness [8] far left; Maximum brightness [9] far
right.
(b) Click video [1], bouncing ball [2] or AVI [3] to select
your input source.
(c) Check the raw video display [10] button.
(d) Set the number of bins [6] you want in your histogram (32-100
is a good number).
(e) Use your mouse to draw a sample box over the colored object
that you want to track.
...Tuning...
(f) Check the histogram "Back Project" [10] button. You
want the object to be well segmented as in figure 1 above. To get
things well segmented, adjust things in the order described below:
(g) Increase (move the slider to the right) the mimimum acceptable
saturation threshold [7]. Make sure you allow a little time to see
how your adjustment is working. Often moving it about 25% of the way to
the right is sufficient.
(h) If (g) doesn't do the whole job, then increase the minimum
acceptable brightness threshold [8] to the right allowing enough time to
see the effect at each position. Usually 20-50% of the way to the right
works.
...not working...
(i) If (g) & (h) do not give you a well segmented object,
you probably need to change your camera settings and/or lighting.
Histogram threshold [5] can have a marginal effect on edge regions.
Decreasing (move to the left) high brightness threshold [9] can diminish
edges and glaring white regions in the video image, usually moving it no
more than 10% of the way to the left. Setting more histogram bins
[6] sometimes helps.
