Temporal Texture and Activity Recognition

Ramprasad Polana and Randal C. Nelson
Department of Computer Science
University of Rochester
Rochester, NY 14627

Abstract: The goal of this thesis is to demonstrate the utility of low-level motion features for the purpose of recognition. Although motion plays an important role in biological recognition tasks, motion recognition, in general, has received little attention in the literature compared to the volume of work on static object recognition. It has been shown that in some cases, motion information alone is sufficient for human visual system to achieve reliable recognition. Previous attempts at duplicating such capability in machine vision have been based on abstract higher-level models of objects, or have required building intermediate representations such as the trajectories of certain feature points of the object. In this work we demonstrate that motion recognition can be accomplished using lower-level motion features, without the use of abstract object models or trajectory representations.

First, we show that certain statistical spatial and temporal features derived from the optic flow field have invariant properties, and can be used to classify regional motion patterns such as ripples on water, fluttering of leaves, and chaotic fluid flow. We then present a novel low-level computational approach for detecting and recognizing temporally repetitive movements, such as those characteristic of walking people, or flying birds, on the basis of the periodic nature of their motion signatures. We demonstrate the techniques on a number of real-world image sequences containing complex non-rigid motion patterns. We also show that the proposed techniques are reliable and efficient by implementing a real-time activity recognition system.