The human side of computer vision

Intelligent agents acting in the real world need advanced vision capabilities to perceive, learn from, reason about and interact with their environment. In this talk, I will explore the role that humans play in the design and deployment of computer vision systems. Large-scale manually labeled datasets have proven instrumental for scaling up visual recognition, but they come at a substantial human cost. I will first briefly talk about strategies for making optimal use of human annotation effort for computer vision progress. However, no dataset can foresee all the visual scenarios that a real-world system might encounter. I will argue that seamlessly integrating human expertise at runtime will become increasingly important for open-world computer vision. I will introduce, and demonstrate the effectiveness of, a rigorous mathematical framework for human-machine collaboration. Looking ahead, in order for such collaborations to become practical, the computer vision algorithms we design will need to be both efficient and interpretable. I will conclude by presenting a new deep reinforcement learning model for human action detection in videos that is efficient, interpretable and more accurate than prior art, opening up new avenues for practical human-in-the-loop exploration.

Olga Russakovsky recently completed her PhD in computer science at Stanford and is now a postdoctoral fellow at Carnegie Mellon University. Her research is in computer vision, closely integrated with machine learning and human-computer interaction. Her work was featured in the New York Times and MIT Technology Review. She served as a Senior Program Committee member for WACV’16, led the ImageNet Large Scale Visual Recognition Challenge effort for two years, and organized multiple workshops and tutorials on large-scale recognition at premier computer vision conferences ICCV’13, ECCV’14, CVPR’15, ICCV’15 and CVPR’16. In addition, she founded and directs the Stanford AI Laboratory’s outreach camp SAILORS (featured in Wired and published in SIGCSE’16) designed to expose high school students in underrepresented populations to the field of AI.