COS 496 - Computer Vision

Spring 2003
Course home Outline and lecture notes Assignments


Assignment 3

Due Monday, Mar. 31


Eigenfaces

Implement a system for recognizing faces using PCA. You might find it interesting to read the Turk and Pentland paper on eigenfaces. This assignment is a bit more open-ended than the previous ones, but at a minimum your system should do the following:

  1. Read a collection of images, and collect them into a big matrix.
  2. Perform PCA with whitening on the collection.
  3. Determine the top k principal components, corresponding to the eigenfaces with maximal variation.
  4. Load a novel image, and, after subtracting off the average, determine its projection onto the basis spanned by the top k principal components.
  5. Use this projection, together with nearest-neighbor search of the k-dimentsional vectors, to guess which face the novel image represents.

You will have to account for the differences in position of the face in each image (a simple way might be to define a standard window size, and center it around a point the user clicks on - save the results so you don't have to do this more than once). Randomly separate the data into training and test data, and see how well you can recognize people.

There are two databases for you to test on: the Yale database (15 people, 11 poses each - available as zip or tar.gz) and the CS496 database (11 people, 2 poses each - zip or tar.gz). If you find additional databases you wish to experiment with, you can of course do that as well.

Benedict has made a cleaned-up version of the face data available:

The Yale faces just have some shadows erased (to white). Our faces have been cropped around the faces, and had the background erased. They have also all been stretched/squished to 150 x 200 pixels. So the heads should line up fairly well.
The Yale face data is available as zip or tar.gz and the CS496 faces are here as zip or tar.gz.


Questions

Answer the following questions, for the Yale database:

  1. How much of the variation is captured by the first 10 principal components? 25?
  2. How many principal components are necessary to obtain human-recognizable reconstructions? That is, if you take an image and project it onto the space spanned by the first k principal components, how big does k have to be before the images look recognizable?
  3. How do the results vary if you do not include the pictures with glasses?
  4. Try some recognition experiments on subsets of the database corresponding to changes in lighting and changes in expression. Which category of variations is easier to accommodate?
  5. Try your recognition algorithm on some non-face images. Can you distinguish between face and non-face images by looking at the difference between the projection of an image onto the space spanned by the first k principal components and the projection onto the orthogonal complement? That is, compare 
    	proj_subspace (image)
    and 
    	image - proj_subspace (image)


Hints

Always remember to subtract the average face and, when reconstructing images based on the first k principal components, add the average back in at the end.

You are not expected to implement SVD from scratch - use the Matlab svd function, or any SVD or eigenvalue code you find.


Submitting

This assignment is due Monday, March 31, 2002 at 11:59 PM Eastern Standard Time. Please see the general notes on submitting your assignments, as well as the late policy and the collaboration policy.

Please submit:

smr@cs.princeton.edu