Overview

In this assignment you will create a simple image processing program. The operations that you implement will mostly be filters that take an input image, process the image, and produce an output image.

Getting Started

You should use the following skeleton code (COS426_assignment1.zip) as a starting point for your assignment. We provide you with several files, but you should mainly change R2Image.cpp.

• imgpro.cpp: Parses the command line arguments, and calls the appropriate image functions.
• R2Image.[cpp/h]: Image processing.
• R2Pixel.[cpp/h]: Pixel processing.
• R2.h: Includes a set of utility code that you may find useful in your assignment (but are not required to use) including...
• R2Line.[cpp/h] Represents a line in 2D
• R2Segment.[cpp/h] Represents a line segment in 2D
• R2Vector.[cpp/h] Represents a vector in 2D
• R2Distance.[cpp/h] Functions to compute distances between geometric objects
• jpeg: A directory with code to read/write JPEG files (used by the framework code in imgpro.cpp, you don't need to use this directly)
• Makefile: Macintosh makefile; used with Unix make
• imagepro.[vcproj/sln]: Project file for Visual Studio .NET 2005 on Windows.

We provide several test images with the skeleton code, however any JPEG, PPM, or 24-bit BMP image should work.

After you copy the provided files to your directory, the first thing to do is compile the program. If you are developing on a Windows machine and have Visual Studio .NET 2005 installed, use the provided project files to build the program. If you are developing on a Mac machine, type make in the assignment1 directory. In either case, an executable called imgpro (or imgpro.exe ) will be created.

Please use either the Visual Studio 2005 or Macintosh make build environments.

How the Program Works

The user interface for this assignment was kept as simple as possible, so you can concentrate on the image processing issues. The program runs on the command line. It reads an image from a file (the first program argument) and writes an image to a file (the second program argument). In between, it processes the image using the filters specified by subsequent command line arguments. For example, to increase the brightness of the image in.bmp by 10%, and save the result in the image out.bmp, you would type:

% imgpro in.bmp out.bmp -brightness 1.1

The brightness filter has already been implemented in R2Image, so you can use the above command line for testing.

For each available image filter there may be one or more optional parameters (e.g., brightness takes a factor). To see the complete list of filters and their parameters, type:

% imgpro -help

If you specify more than one filter on the command line, they are applied in the order that they are found. For example,

% imgpro in.bmp out.bmp -contrast 0.8 -scale 0.5 0.5

would first decrease the contrast of the input image by 20%, and then scale down the result by 50% in both x and y directions. Of course, you are welcome to create your own filters and add program arguments for them by editing imgpro.cpp. However, please do not change the program arguments for the filters already provided.

What You Have to Do

The assignment is worth 20 points. The following is a list of features that you may implement (listed roughly from easiest to hardest). The number in front of the feature corresponds to how many points the feature is worth. The features in bold face are required. The other ones are optional. For each group of features (e.g., Per-pixel Operations), a minimum and maximum number of possible points is listed -- you will only get credit for features implemented within each section up to the maximum listed. Refer to the examples web page for more details on the implementation of each filter and example output images.

• Per-pixel Operations. (1pt minimum, 5pts maximum)
  • (1/2) Random noise: Add noise to each pixel of an image.
  • (1/2) Speckle: Randomly set some pixels to a random color. This is different from the previous in that only certain, randomly chosen pixels are modified, and set to a random value, while the previous modifies all pixels by a limited amount.
  • (1/2) Saturation: Change the saturation of an image. See Graphica Obscura.
  • (1/2) Contrast: Change the contrast of an image. See Graphica Obscura.
  • (1/2) Threshold: Set values below the threshold to zero
  • (1/2) Gamma: Apply a gamma correction to the image.
  • (1/2) Black and White: Convert to gray levels by replacing each pixel with its luminance.
  • (1/2) Extract Channel: Leave specified channel intact and set all others to zero.
  • (1) Quantize: Change the number of bits per channel of an image, using simple rounding.
• Linear Filtering Operations (1pt minimum, 4pts maximum)
  • (1) Gaussian Blur: Blur an image by convolving it with a Gaussian low-pass filter.
  • (1) Sharpen: Apply a linear sharpening filter to the image
  • (1) Edge detect: Detect edges in an image by convolving it with an edge detection kernel.
  • (1) Motion Blur: Apply a left-to-right motion blur of the image
• Non-Linear Filtering Operations (2pts minimum, 8pts maximum)
  • (2) Median Filter: Remove speckle noise using a median filter of given width
  • (2) Bilateral Filter: Smooth while preserving sharp edges in the original. See here.
  • (2) Wavelet transform: Transform an image to the Haar wavelet domain. See Wavelets for Computer Graphics.
  • (2) Wavelet inverse transform: Transform an image from the Haar wavelet domain
• Dithering Operations (2pts minimum, 5pts maximum)
  • (1) Random dither: Convert an image to a given number of bits per channel, using a random threshold.
  • (2) Ordered dither: Convert an image to a given number of bits per channel, using a 4x4 ordered dithering matrix.
  • (2) Floyd-Steinberg dither: Convert an image to a given number of bits per channel, using dithering with error diffusion.
• Resampling Operations (4pts minimum, 6pts maximum) - You must provide methods for point, linear, and gaussian sampling)
  • (2) Scale: Scale an image up or down by a real valued factor.
  • (2) Rotate: Rotate an image by a given angle.
  • (2) Fun: Warp an image using a non-linear mapping of your choice (examples are fisheye, sine, bulge, swirl).
    Clarification: Implement all three sampling methods, which can be controlled by the -sampling option prior to -scale, -rotate, or -fun. Using Gaussian sampling with a fixed filter sigma will gain full credit; setting the filter sigma based on the amount of scaling (keep it fixed for rotate and fun) will gain you 1 point extra credit.
• Miscellaneous (8pts maximum)
  • (1) Crop: Given an x/y origin and a width/height value in pixels, crop the input image.
  • (1) Composite: Compose one image with a second image, using a third image as a matte.
  • (3) Morph: Use pairs of corresponding lines to morph a source image into a target image using a morph parameter t. See [Beier92].
  • (up to 3) Nonphotorealism: Implement any non-trivial painterly filter. For inspiration, take a look at the effects available in programs like xv, PhotoShop, and Image Composer (e.g., impressionist, charcoal, stained glass, etc.). The points awarded for this feature will depend on the creativity and difficulty of the filter. At most one such filter will receive points.
  • (up to 3) Anything else: Implement any non-trivial image processing algorithm of your own choosing, and we will give you points according to the difficulty.

By implementing all the required features, you get 10 points. There are many ways to get more points:

• implementing the optional features listed above;
• (1) submitting one or more images for the art contest,
• (1) submitting a .mpeg or .avi movie animating the results of one or more filters with continuously varying parameters (e.g. the morph),
• (2) winning the art contest.

For images or movies that you submit, you also have to submit the sequence of commands used to created them, otherwise they will not be considered valid.

It is possible to get more than 20 points. However, after 20 points, each point is divided by 2, and after 22 points, each point is divided by 4. If your raw score is 19, your final score will be 19. If the raw score is 23, you'll get 21.25. For a raw score of 26, you'll get 22.

Extra credit points cannot replace the required features (bold items). Your final score will be calculated by adding 10 to the number of extra credit points you achieve, applying the above formula for diminishing returns, and then subtracting the number of required points that you missed.

What to Submit

You should submit one archive (zip or tar file) containing:

• the complete source code, along with an updated Visual Studio project file or Makefile (if you added any source files)
  
• the .mpeg or .avi movie for the movie feature (optional);
• the images for the art contest (optional); and
• a writeup.

The writeup should be a HTML document called assignment1.html which may include other documents or pictures. It should be brief, describing what you have implemented, what works and what doesn't, how you created the composite image, how you created the art contest images, and instructions on how to run the fun filters you have implemented.

We will not attempt to grade assignments which do not build in Visual Studio 2005 or Mac OS X.

NOTE: Please convert your art contest and writeup images to JPEG format before submitting them (to save space). You can use your program with no filters to do the conversion (e.g., imgpro in.bmp out.jpg).

Should you choose to create a video of your morph there are many options for video encoders available, however we have found that FFMPEG works well and runs on both Windows and linux. MEncoder is another option.

Always remember the late policy and the collaboration policy.

Hints

A few hints:

• Do the simplest filters first!
• Look at the example page.
• There are functions to manipulate pixel components and pixels in R2Image.[cpp/h]. You may find them helpful while writing your filters.
• There are functions to manipulate 2D geometric primitives in R2Distance.[cpp/h], R2Segment.[cpp/h], R2Line.[cpp/h], R2Point.[cpp/h], and R2Vector.[cpp/h]. You may find them helpful while writing the morphing functions.
• Send mail to the cos426 mailing list.
• The code should compile and link as-is (make sure to try it out before adding your own code!). If you have problems linking to libjpeg, you can simply remove the -DUSE_JPEG flag from the makefile (or #undef USE_JPEG in your code) to disable JPEG support.
• If you should happen stumble upon anything that you suspect is a bug in the framework code (we're not perfect either!) please mail the preceptor as soon as possible so that it can be corrected.
• NEW: A very useful example site can be found here

FAQ

• The compose image parameters don't make sense. Why?
  Unfortunately, the image file formats (bmp, jpg, and ppm) don't store alpha values, so we need a way of getting alpha values for both the top and bottom images. The way we are doing this is by loading a second image for each and treating its blue channel as the alpha value. So... we need 4 images: bottom, bottom_mask, top, and top_mask. bottom is the current image that's already loaded, so the ComposeImage option takes the remaining 3 filenames as input: top, bottom_mask, and top_mask. In main.cpp the 4 images are put together to create two images with the alpha values filled in, which are then passed to the Composite function that you write in R2Image.cpp. 

• For the quantization function, should we assume that the number of bits to which we're quantizing is always a multiple of 3?
  No. The parameter is the number of bits per channel, not per pixel.

• I get an assertion error messages and a core dump when the program tries to read in my bmp files. What should I do?
  Make sure your files are 24-bit, uncompressed bitmaps. The provided code does not read any other kind of .bmp files. You can convert an image to the right .bmp format using ImageMagick (this is installed on most of the Princeton Unix machines you can SSH to), or use your favorite paint program to save the file as uncompressed BMP or JPEG..

• When doing convolutions, each pixel becomes a weighted average of its neighbors. What do I do when the pixel is close to the border, and therefore doesn't have some of its neighbors?
  There are a few things you could do. First of all, here is what not to do: don't just skip those pixels. One thing you could do is pretend that the image is "toroidal" -- the right side is connected to the left side, and the top to the bottom. So, the pixel just to the right of the rightmost pixel is the leftmost pixel. Another thing you could do is adjust the convolution matrix for the edge and corner points. When you adjust the matrix, take care that the entries in the matrix still sum to 1.0. For example, if you are applying the edge detection matrix

  |-1 -1 -1|
  |-1 +8 -1|
  |-1 -1 -1|

  to a pixel in the middle of the right column of pixels, you would use

  |-1 -1 0|
  |-1 +5 0|
  |-1 -1 0|
  
  In Floyd-Steinberg, you won't be able to transfer any error from the bottom row, but this doesn't matter.

• Is it ok to just do nothing if the filter argument in a convolution is larger than the size of the image.
  Yes, but a warning message would be nice.

• My blur filter is darkening the image. What is wrong?
  Make sure that the entries in your kernel add up to 1. Be especially careful with rounding errors: although small in each entry, collectively they can be large enough to cause this undesired darkening effect.

• Can I assume my functions will always be called with valid arguments?
  No, but bailing out is acceptable (print a message and exit). Please avoid segmentation faults, core dumps, blue screens of death, etc.