COS 429 - Computer Vision

Spring 2004
Course home Outline and lecture notes Assignments


Assignment 3

Due Thursday, Apr. 1


Eigenfaces

Implement a system for recognizing faces using PCA. You might find it interesting to read the Turk and Pentland paper on eigenfaces. This assignment is a bit more open-ended than the previous ones, but at a minimum your system should do the following:

  1. Read a collection of images, and collect them into a big matrix.
  2. Perform PCA with whitening on the collection.
  3. Determine the top k principal components, corresponding to the eigenfaces with maximal variation.
  4. Load a novel image, and, after subtracting off the average, determine its projection onto the basis spanned by the top k principal components.
  5. Use this projection, together with nearest-neighbor search of the k-dimentsional vectors, to guess which face the novel image represents.

You will have to account for the differences in position of the face in each image (a simple way might be to define a standard window size, and center it around a point the user clicks on - save the results so you don't have to do this more than once). Randomly separate the data into training and test data, and see how well you can recognize people.

There are two databases for you to test on. The first is a collection of faces from researchers at Yale (15 people, 11 poses each - available as zip or tar.gz). There is also a cleaned up version of this database with some shadows erased (avaliable as zip or tar.gz - thanks to Benedict Brown).

The second is a group of pictures of the people in this class (available as zip). If you find additional databases you wish to experiment with, you can of course do that as well.


Questions

Answer the following questions, for the Yale database:

  1. How much of the variation is captured by the first 10 principal components? 25?
  2. How many principal components are necessary to obtain human-recognizable reconstructions? That is, if you take an image and project it onto the space spanned by the first k principal components, how big does k have to be before the images look recognizable?
  3. How do the results vary if you do not include the pictures with glasses?
  4. Try some recognition experiments on subsets of the database corresponding to changes in lighting and changes in expression. Which category of variations is easier to accommodate?
  5. Try your recognition algorithm on some non-face images. Can you distinguish between face and non-face images by looking at the difference between the projection of an image onto the space spanned by the first k principal components and the projection onto the orthogonal complement? That is, compare 
    	proj_subspace (image)
    and 
    	image - proj_subspace (image)


Hints

Always remember to subtract the average face and, when reconstructing images based on the first k principal components, add the average back in at the end.

You are not expected to implement SVD from scratch - use the Matlab svd function, or any SVD or eigenvalue code you find.


Submitting

This assignment is due Thursday, April 1. Please see the general notes on submitting your assignments, as well as the late policy and the collaboration policy.

Please submit: