Perceptual Data Mining

Imagine if computers possessed the ability of small children to observe and comprehend their dynamic environment. Very young children can track moving objects, differentiate them, identify them, understand their activity, and understand their interactions with static objects and with other moving objects in the world. There are amazing benefits in coupling these basic human abilities with the unique capabilities of computers: to communicate instantly with high bandwidth; to store and index massive quantities of observations with perfect recall; to process in parallel; and to draw inferences over extremely large stores of data. These benefits have driven the increased interest in automated computer vision applications, such as intelligent visual surveillance, automated traffic analysis, quantitative experimental animal observation, and wide-area scene understanding enabling high-level computer vision research that are predicated on the ability to detect and localize certain types of objects.

This talk describes my work in Perceptual Data Mining (PDM), a bottom-up data-driven framework for bootstrapping visual intelligence in novel environments. This work is focused on developing computational analogs for basic human perception and exploiting the strengths of computers to take full advantage of these capabilities. This research has centered on the development of systems that are capable of: automatically tracking multiple objects in real-time across multiple overlapping and non-overlapping cameras in unstructured indoor and outdoor environments; automatically modeling the types of objects in a particular environment; automatically modeling the activities that these objects perform; learning patterns of the activities over extended periods of time; and detecting unusual objects or behavior. Even without supervision, this system can create a compact description of the objects and activities in an environment that enables effective query retrieval. With minimal supervision, this system can communicate and summarize the activity in an environment. More information: http://www.csail.mit.edu/~stauffer/