How the System Works:
As shown in the data-flow diagram (above),
the steps in the system are, from top to bottom:
The entire process is controlled from a
shell script that executes programs to
compute the optical flow, calculate the scalar shape descriptions,
and analyze the signals for frequency and phase.
- The system begins with a motion sequence of n+1
frames of a person walking.
- We compute the optical flow
(Bulthoff, Little, and Poggio -
of the motion sequence to get n
frames of (u,v) data, where u is the
x-direction flow and v is the
View MPEGs of flow for sample sequence:
In the u and v images, red indicates negative
flow while green indicates positive. To save computation time,
we manually tracked each subject and computed flow only in a
box surrounding the person. You can see artifacts from this
box in the MPEG images of the flow sequences.
- Spurious motion in the image background can confound recognition.
The aspect ratio signals are particularly sensitive
to outliers. To eliminate as many outliers as possible, we
compute connected components of regions that are moving
and discard the small components. The resulting moving
"blobs" have less spurious motion.
- For each frame of the flow, we compute a set of scalars
(see source code) that
characterize the shape of the flow in that frame. Examples
include x and y coordinates of the moving region
(centx and centy),
and the aspect ratio of the moving region (aspct).
View the cropped motion sequence with the
the position of (centx,centy) shown as a "+",
the |(u,v)|-weighted centroid
(wcentx,wcenty) shown as a box,
a solid ellipse corresponding to aspct,
and a dashed ellipse
corresponding to the aspect ratio of the
moving region, waspct.
Note how the weighted ellipse (particulary senstive
to fast-moving parts of the body) follows the motion of
the feet as they accelerate in the gait.
- We rearrange the scalars to form one time series for each
View plots of some of the signals for the example sequence:
- The time series are sinusoidal and have a common frequency.
We use least-squares linear prediction spectrum analysis
(Barrodale and Erikson's method -
to find the fundamental frequency and phase for each series.
A Matlab example that demonstrates
the advantages of this method is available.
View plots of the spectrum analysis for the example
- Phase values are arbitrary and depend on the point in the gait
at which the image sequence begins. We select a single signal
to be a reference for the others. Phases, with the reference
phase subtracted, are features.
- Selected features combine to form a feature vector for recognition.
The plot shows the average feature values and the
maximum and minimum values for each subject in our