Predicting activities in images

The task of visual prediction is important for two main reasons: (a) For intelligent agents and systems, prediction is vital for decision making. For example, in order to perform assistive activities, robots must be able to predict the intentions of other agents in the scene. Even a task as simple as walking through a crowded hallway requires the prediction of human trajectories. (b) More importantly, prediction requires deep understanding of the visual world and complex interplay between different elements of the scene. Therefore, prediction can act as a way to define “what does it mean to understand an image,” and the task of visual prediction can act as an enabler for scene understanding. In this talk, I will go over several techniques for prediction developed over the past few years. These techniques operates at different levels of descriptions from high-level object motions, to mid-level patch motion and appearance changes, to low-level optical flow. They operate at different levels of supervision from strong the supervision requiring explicit object tracking to unsupervised learning for patch-based prediction. I will discuss the pros and cons of different levels, ideas for combining them and example applications in robotics and transportation applications.

Martial Hebert is a Professor of Robotics Carnegie Mellon University and Director of the Robotics Institute, which he joined in 1984. His interest includes computer vision, especially recognition in images and video data, model building and object recognition from 3D data, and perception for mobile robots and for intelligent vehicles. His group has developed approaches for object recognition and scene analysis in images, 3D point clouds, and video sequences. In the area of machine perception for robotics, his group has developed techniques for people detection, tracking, and prediction, and for understanding the environment of ground vehicles from sensor data. He has served on the editorial boards the IEEE Transactions on Robotics and Automation, the IEEE transactions on Pattern Analysis and Machine Intelligence, and the International Journal of Computer Vision (for which he currently serves as Editor-in-Chief).