COS 429 - Computer Vision

Fall 2009

Assignment 1

0. Getting familiar with Matlab

Matlab is available on several OIT machines (look here for details). You should be able run it remotely from any Unix workstation.

Alternatively, you can install it on your own matchine - look here for details.

Read through the following for a basic introduction to Matlab:

• A short Matlab introduction
• The demos accessible by typing "demo" within Matlab.

Work through the following tasks using an image of your choice. You do not need to submit any results, but make sure you are comfortable doing the following:

1. Read an image into a variable.
   Hint #1: "help imread"
   Hint #2: Use single quotes around the filename.
   Hint #3: Ending a command with a semicolon supresses printing the result.
   
   
2. Display the image.
   Hint: "help image"
   
   
3. Adjust the axes so that the aspect ratio is 1:1 (i.e., "square pixels").
   Hint: "help axis"
   
   
4. Convert the image to grayscale.
   Hint: "help rgb2gray"

   Note: if your version of Matlab doesn't have the rgb2gray function, download rgb2gray.m. Place this in your working directory, and it should be auto-loaded by Matlab.

5. Convert the grayscale image to floating point.
   Hint: "help double"
   
   
6. Display the floating-point image, using a grayscale colormap.
   Hint #1: "colormap(gray)"
   Hint #2: "help imagesc"
   
   
7. Plot the intensities along one scanline of the image.
   Hint #1: Extracting a part of a matrix is done by

   	matrix2 = matrix1(row_min:row_max,col_min:col_max);
   

   Indices in Matlab are 1-based (not 0-based as in C).
   row_max or col_max may be "end" to indicate the last element.
   Just a ":" is equivalent to "1:end".
   Hint #2: "help plot"
   
   
8. Store the width and height of the image in variables "width" and "height".
   Hint #1: "help size"
   Hint #2: Functions in Matlab may return multiple values. You can get at the values using the notation

   	[var1, var2] = func(x)
   

   Hint #3: In Matlab, the number of rows is the first dimension and the number of columns is the second.
   
   
9. Write a pair of nested "for" loops to set a grid of every 10th pixel horizontally and every 20th pixel vertically to 0.
   Hint #1: "help for"
   Hint #2: "start:increment:stop"
   
   
10. Create a function "maxrow" that takes a matrix and a row index and returns the brightest pixel in the given row. Store the function in a file called "maxrow.m" so that Matlab loads it automatically when you call the function.
    Hint #1: "help function"
    Hint #2: Matlab has many built-in functions that operate on entire vectors or matrices. There might be one to compute the maximum...
    
    
11. Write the modified image back to a new file.
    Hint #1: "help uint8" to convert the image back to integer
    Hint #2: "help imwrite"
    
    

If you get stuck on any of these, feel free to ask for help, either by emailing cos429 at lists cs princeton edu or by asking a colleague.

1. Canny edge detector (50%)

• Filtered gradient:
  1. Load an image
  2. Find the x and y components of the gradient Fx and Fy of the image smoothed with a Gaussian.
  3. Compute the edge strength F (the magnitude of the gradient) and edge orientation D = arctan(Fy/Fx) at each pixel.
  Hint #1: Recall that a 2D Gaussian is separable, and that finding the derivative of a function convolved with a Gaussian is the same as convolving with the derivative of a Gaussian. You should use these facts in your code.
  Hint #2: "help conv2", especially the syntax for independently convolving rows and columns.
  
  
• Nonmaximum suppression:
  Create a "thinned edge image" I(x,y) as follows:
  1. For each pixel, find the direction D* in (0, 45, 90, 135) that is closest to the orientation D at that pixel. (Or the equivalent in radians...)
  2. If the edge strength F(x,y) is smaller than at least one of its neighbors along D*, set I(x,y) = 0, else set I(x,y) = F(x,y).
  
  
• Hysteresis thresholding:
  Repeatedly do the following:
  1. Locate the next unvisited pixel (x,y) such that I(x,y) > T_h.
  2. Starting from (x,y), follow the chain of connected local maxima, in both directions, as long as I(x,y) > T_l.
  3. Mark each pixel as it is visited.
  
  
• Edge image:
  Create an image with all edge pixels marked in white, and all non-edges in black.

Test your alogrithm on images of your choosing, experimenting with different values of the parameters sigma (the width of the Gaussian used for smoothing), T_h (the "high" threshold), and T_l (the "low" threshold). Also run your algorithm on the following images:

• mandrill.jpg: Try different parameter values.
• csbldg.jpg: Try to find values that will find just the outline of the building, and others that will find edges between individual bricks.

2. Corner detector (25%)

• Filtered gradient: Same as above for the Canny edge detector.
• Finding corners:
  1. Compute the covariance matrix C over a neighborhood around each point.
  2. Compute the smaller eigenvalue of C.
     Hint: "help eig"
  3. Save all points at which the smaller eigenvalue l2 is greater than a threshold into a list L.
• Nonmaximum suppresion:
  1. Sort L in decreasing order of l2.
     Hint: "help sortrows"
  2. For each point p, remove all points in the neighborhood of p that occur lower in L.
• Marking corners: Within the original image, draw a small square centered at the location of each corner.

Test your algorithm on the building image above, and on checker.jpg. (This is a picture of a target used for camera calibration. Note the distortion in this lens - is it barrel or pincushion?) Explore the effects of changing sigma (the width of the Gaussian), the size of the neighborhood, and the threshold.

3. Scale selection or "blob detection" (25%)

Find features that are maxima of a Difference of Gaussians (DoG), over scale and position:

• Multiscale filtering: Compute versions of the image filtered by Gaussians with sigma = 1, sqrt(2), 2, 2 * sqrt(2), 4, etc.
• DoG: Compute differences between images filtered with each pair of adjacent scales
• Scale selection: Compare each DoG pixel to its 26 neighbors: all neighbors in space, and at both adjacent scales. Mark as candidate features those pixels that are either larger or smaller than all of their neighbors. It is also acceptable to keep only the global maximum over scale, as long as it is a local maximum among its spatial neighbors.
• Threshold: Keep only points having a DoG value whose absolute value is above some threshold.
• Marking corners: Within the original image, draw a square centered at the location of each detected feature point, with a size equal to the detected scale.

Test your algorithm on an image of your choice.

4. Extra credit (max 5% extra)

Experiment with heuristics for automatically determining the thresholds to be used for each image. Some of these are mentioned in Trucco and Verri, but feel free to be creative and come up with your own.

Submitting

This assignment is due Tuesday, October 6, 2009 at 11:59 PM. Please see the general notes on submitting your assignments, as well as the late policy and the collaboration policy.

Please submit:

• Your edge, corner, and blob detection code (as one or more .m files).
• For each test image, images showing the smoothed horizontal and vertical gradient, the gradient magnitude, and the gradient magnitude after nonmaximum suppression.
• The results of your edge, corner, and blob detectors on the provided sample images (using the best parameters you were able to find).
• A README or README.html file containing a description of your experiments with different parameters, and any relevant implementation notes.

Note that programming in Matlab is not an excuse to write unreadable code. You are expected to use good programming style, including meaningful variable names, a comment or three describing what the code is doing, etc. Also, all images you submit should have been saved with the "imwrite" function - do not submit screen captures of the image window.