COS 429: Computer Vision, Fall 2013
Assignment 0: Warm Up

Part 1: Thought Exercise

The first objective of this assignment is to get you thinking about the future implications of computer vision on society.

Imagine in the future that every person and object in the world is equipped with a "COS429 video camera" that can capture images with 10⁷ pixels at 10³ frames per second continuously, and that storage and processing on the camera is capable enough to analyze each image, adapt camera settings, store images indefinitely, and communicate with the cloud in real-time.

Please submit a writeup (see below) where the first section contains at most one paragraph (less than 1/2 page) describing at least one new application enabled by computer vision using this "COS429 video camera." Note: there is no "right" answer -- wild and creative ideas are encouraged.

Part 2: Programming Exercise

The second objective of the assignment is to get familiar with programming in MATLAB.

If you have no MATLAB experience (probably most of you), we suggest that you follow this tutorial to learn the basics of how to dowload MATLAB and begin programming in it. We will be holding an optional precept as an introduction to MATLAB on Friday, September 13 at 3:00PM in CS 105 (the small auditorium). And/or, you can attend the MATLAB Short Course offered by the Keller Center during Monday and Tuesday evenings (September 16-17) in Friend 101.

Your goal for this assignment is to write a MATLAB program, detecteyes.m, that predicts the locations of people's eyes in an input image. Specifically, "output_image = detecteyes(input_image)" should return a gray-level output image where the brightness of each pixel is proportional to the predicted probability of finding a person's eye at that pixel in the given input_image. For example, running detecteyes for the input image shown on the left below might produce the output shown in the middle. The rightmost image shows an overlay of the two (where the output has been added to the red channel of the input).

Input Output Overlay

This is just a warmup exercise. So, you can use any strategy you wish for the core algorithm in your implementation of detecteyes. Simple methods are expected -- just experiment with the matrix and image manipulation functions provided by MATLAB, and then choose a strategy that you think will produce a good result. We do not expect you to spend more than a couple of hours on the final implementation once you are familiar with MATLAB.

The focus of this part of the assignment is mainly to help you get used to implementing and running image analysis programs in MATLAB, not on designing sophisticated image analysis algorithms (that comes later in the course). So, you will not be graded on the prediction accuracy of your program. Really. Not at all. If you are a novice and the best you can do is create a random pattern of black and white pixels, that is OK -- you will get full credit for the programming part of the assignment as long as your program produces an output image different than the input. In any case, in the second section of your writeup (see below), please include a section titled "Programming Exercise" with a description of what algorithmic strategy you chose to implement and why you chose it.

Part 3: Results and Analysis

The third objective of the assignment is to get you thinking about how to evaluate algorithms in computer vision. Ideally, all programs should work perfectly in all cases ... but surely they will not. So, understanding when, how well, and why your program works is an important part of any programming task, especially in computer vision.

You should execute your detecteyes program on a variety of input test images to investigate how well it works under different conditions. To get full credit for this part, you must show outputs of your program for all of the images in the "input" subdirectory of cos429_assignment0.zip, plus at least three images taken with your own camera, and at least three images downloaded from the web (e.g., from here).

When choosing your own input images, please think about how to investigate how well your algorithm works. For example, you should show the most difficult case(s) your algorithm can handle successfully, plus some that it cannot. You should show sequences of images demonstrating the impact of varying at least one image property on your output (e.g., image resolution, sensor noise, scene lighting, camera viewpoint, background clutter, facial features, etc.). You could also show images demonstrating the impact of parameter settings in your program (e.g., the sigma used for a blur).

Please include the results of your program on the test data set and your own input images in a third section of your writeup titled "Results and Analysis." For each input image (or group of images), you should include a short discussion of how your algorithm works on that image. Overall, the goal of this section is to answer questions like: When does your program succeed? When does it fail? What are the key attributes of input images that affect its success? What parameter settings affect its success? This is a warmup exercise, and so your analysis does not have to be complete. However, it should discuss at least one characteristic of your input images that affects the quality of your results, demonstrated by results for a set of input images spanning cases where the algorithm does and doesn't work well.

Getting Started

You should start by downloading cos429_assignment0.zip, which provides: 1) the directory structure to follow in your submission, 2) a template for your writeup (writeup/writeup.html), 3) a set of test input images (input/test*.jpg), 4) a simple implementation of detecteyes (code/detecteyes.m), and 4) a script to ease reading, writing, converting, and creating images for your writeup (code/runme.m).

You should edit detecteyes.m to implement your algorithm, download new test images into the input subdirectory, execute runme.m to produce output and overlay images for all inputs, and complete your writeup.

Submitting Your Solution

Please submit your solution via the dropbox link here.

Your submission should include a single file named "assignment0.zip" with the following structure:

A subdirectory named "code" containing your source code in one or more MATLAB file. Your main program should be in a file called detecteyes.m. A script to run your program on all inputs is in a file called runme.m (you should not have to edit it, but you can if you want).
A subdirectory named "input" containing all test input files in ".jpg" format.
A subdirectory named "output" containing the images produced by running your program on each of the input images (should be produced automatically by runme.m). Please include all output files in .jpg format.
A subdirectory named "overlay" containing the images produced by overlaying an output image over its corresponding input (should be produced authomatically by runme.m). Please include all overlay images in .jpg format.
A subdirectory named "writeup" containing a file named "writeup.html" containing separate sections titled as followed (you can start from the template provided in the .zip file):
1. Thought exercise: your answer to the question in Part 1.
2. Programming exercise: a description of the algorithmic strategy used in your program of Part 2, along with an explanation of why you chose it.
3. Results and Analysis: display and discussion of images produced by your program for different inputs and/or parameter settings.
4. Conclusion: summary of your findings.
5. Notes: any other comments, description of any assistance received and/or any discussions had with other students.

Please follow the general policies for submitting assignments, including the late policy and collaboration policy.