COS 426: Computer Graphics Spring 2014

Overview

In this assignment you will create a simple image processing program. The operations that you implement will mostly be filters that take an input image, process the image, and produce an output image.

Running the Program

The image processing program, imgpro, runs on the command line. It reads an image from a file (the first program argument) and writes an image to a file (the second program argument). In between, it processes the image using the filters specified by subsequent command line arguments. For example, to add 50% random noise to an image in.jpg and save the result in the image out.jpg, you would type the following command (the noise filter has already been implemented, so you can test this command right away):

% imgpro in.jpg out.jpg -noise 0.5

For each available image filter there may be one or more optional parameters (e.g., noise takes a magnitude). To see the complete list of filters and their parameters, type:

% imgpro -help

If you specify more than one filter on the command line, they are applied in the order that they are found. For example,

% imgpro in.jpg out.jpg -contrast 1.2 -scale 0.5 0.5

would first increase the contrast of the input image by 20%, and then scale down the resolution of the result by 50% in both x and y directions. Of course, you are welcome to create your own filters and add program arguments for them by editing imgpro.cpp. However, please do not change the program arguments for the filters already provided.

Implementing the Image Processing Filters

The assignment is worth 20 points. The following is a list of features that you may implement (listed roughly from easiest to hardest). The number in front of the feature corresponds to how many points the feature is worth. The features in bold face are required. The other ones are optional. Refer to the examples web page for more details on the implementation of each filter and example output images.

• Luminance Operations
  • (1/2) Brightness: Change the brightness of an image by scaling RGB values by a factor.
  • (1/2) Contrast: Change the contrast of an image using the method described in Graphica Obscura.
  • (1/2) Gamma: Apply a gamma correction to the image.
  • (1/2) Vignette: Apply a vignetting operator to the image.
  • (3) Histogram equalization: Increase the contrast of the image by histogram equalization.
• Color Operations
  • (1/2) Black & White: Convert to gray levels by replacing each pixel with its luminance.
  • (1/2) Saturation: Change the color saturation of an image using the method described in Graphica Obscura.
  • (2) White balance: Remap the colors so that a given RGB value becomes white using Von Kries' method.
• Linear Filtering Operations
  • (1/2) Blur: Blur an image by convolving it with a Gaussian low-pass filter.
  • (1/2) Sharpen: Apply a linear sharpening filter to the image.
  • (1/2) Edge detect: Detect edges in an image by convolving it with an edge detection kernel.
  • (1) Convolve: Convolve an image with a filter read from an image file.
• Non-Linear Filtering Operations
  • (1) Median Filter: Remove speckle noise using a median filter of given width
  • (2) Bilateral Filter: Smooth while preserving sharp edges in the original. See here or here.
• Dithering Operations
  • (1/2) Quantize: Change the number of bits per channel of an image, using simple rounding.
  • (1) Random dither: Convert an image to a given number of bits per channel, using a random threshold.
  • (2) Ordered dither: Convert an image to a given number of bits per channel, using a 4x4 ordered dithering matrix.
  • (2) Floyd-Steinberg dither: Convert an image to a given number of bits per channel, using dithering with error diffusion.
• Resampling Operations
  • (1) Scale: Scale an image up or down in resolution by a real valued factor.
  • (2) Rotate: Rotate an image by a given angle.
  • (2) Fun: Warp an image using a non-linear mapping of your choice (examples are fisheye, sine, bulge, swirl).
• Composite Operations
  • (1) Composite: Compose one image with a second image using the over operator, with a third image as a matte (alpha).
  • (3) Morph: Use pairs of corresponding lines to morph a source image into a target image using a morph parameter t. See [Beier92]. Use the provided line correspondence editor to define the line correspondences.
  • (3) Fuse exposures: Combine multiple images taken with different exposures using the method described here and demonstrate the result for images taken with different apertures, depths of field, or any other camera setting.
• Miscellaneous
  • (up to 3) Nonphotorealism: Implement any non-trivial painterly filter. For inspiration, take a look at the effects available in programs like xv, PhotoShop, and Image Composer, or GIMP (e.g., impressionist, charcoal, stained glass, etc.). The points awarded for this feature will depend on the creativity and difficulty of the filter. At most one such filter will receive points.
  • (up to 3) Anything else: Implement any non-trivial image processing algorithm of your own choosing, and we will give you points according to the difficulty. Note: you may get credit for this feature only once.
  • (5) Seam carving: Reduce the resolution of an image using the seam carving algorithm described here.

By implementing all the required bold features, you get 13 points. There are many ways to get more points:

• Implementing the optional features listed above;
• (1) Submitting a movie animating the results of apply one or more filters with continuously varying parameters (e.g. the morph);
• (1) Submitting one or more images or movies for the art contest; and
• (2) Winning the art contest.

To get full credit for any of the operations requiring resampling (scale, rotate, fun, morph), you must support three different sampling methods (point, linear, and Gaussian) and compare the results for at least one example. The sampling method to be used by imgpro is controlled by arguments specified before the resampling operation on the command line. For example, imgpro in.jpg out.jpg -point_sampling -rotate 1 -gaussian_sampling rotate 3 will rotate the input image by 1 radian using point sampling and then rotate the result by 3 radians using Gaussian sampling. For Gaussian sampling, choose sensible Gaussian filter dimensions. For scaling, this requires adapting the extent of the Gaussian, for rotation and fun, you may use a fixed size of your choosing.

It is possible to get more than 20 points for this assignment. Your final score will be calculated by adding your score on the required features (up to 13 points) and your bonus for winning the art contest (up to 2 points) to your score on the non-required features, where the value of non-required features incurs diminishing returns after the first 7 points: each successive point beyond 7 is worth 2/3 as much as the previous one. For example, if you get 12 of the 13 points for required features listed in bold and correctly implement optional features worth 10 more points, then your score is 20.4 (12 + 7 + 2/3 + (2/3)^2 + (2/3)^3).

Getting Started

You should download the skeleton code from (cos426_assignment1.zip) as a starting point for the assignment. The zip file contains the correct directory structure for submissions, helpful source code, and a template writeup html file (see cos426_assignment1/README.txt in the zip file for details). To implement the image processing features listed above, you should add code to the appropriate functions within src/R2Image.cpp, add commands to the Makefile to test them, and add sections to the writeup.html to present and discuss your results.

Submitting

You should submit your solution using one zip file named cos426_assignment1.zip via CS dropbox at this submission link. The submitted zip file should have following internal directory structure:

cos426_assignment1/

• writeup.html - your writeup (see the description below),
• Makefile - a Linux script that generates all the images in the output directory from the data in the input directory)
• src/ - the complete source code, including all the code for libraries, compilation, etc.
• input/ - all the input data for the commands in your Makefile
• output/ - all the output images referenced by your writeup
• art/ - all images submitted for the art contest (optional), and movie files submitted for the movie feature (optional).

The writeup.html file should be an HTML document demonstrating the effects of the features you have implemented and would like scored. For some features (e.g., black & white), you can simply show the input and output of your program. However, for features that take an input parameter (e.g., blur), you should provide a series of images showing at least three settings of the input parameter to demonstrate that your code is working properly. You can start from the writeup.html provided with the distribution as a template -- simply add sections to that HTML file for the features you implement in the following format:

• the name of the feature on the top line,
• the input image and output image(s) on a second line,
• text listing the command(s) used to generate the output images on the following lines, and
• (optionally) comments about what to look for in your output (including possible problems in the implementation).

The src directory should have all code required to compile and link your program (including the files provided with the assignment), along with a Visual Studio Solution file and a Makefile to rebuild the code. We will not attempt to grade assignments that neither build in Visual Studio nor with GNU Make under Mac OS X.

The Makefile should be a script containing the commands that generate all the images in the output directory that demonstrate features of your program you would like scored. It should run without crashing when started in the cos426_assignment1/ directory (i.e., it should be possible to delete all the files in the output directory and then run make to regenerate them).

We will test your code on Linux by running make in your src/ subdirectory, followed by make in the main assignment directory to create the output images. Please ensure that your code builds/runs in this way before submitting. Your Makefile should generate all the images in the output directory referenced by your writeup.

To save space, please submit images in .jpeg format (not .bmp or .ppm) and remove binaries and backup files from the src directory before submitting -- i.e., run make clean (under Linux or Mac OS) and execute "Clean Solution" on the "Build menu" in MS Visual Studio.

Note that you are expected to use good programming style at all times, including meaningful variable names, a comment or three describing what the code is doing, etc. Partial credit may not be assigned for code without comments.

Hints

A few hints:

• Do the simplest filters first!
• Look at the example page.
• There are functions to manipulate pixel components and pixels in R2Image.[cpp/h]. You may find them helpful while writing your filters.
• There are functions to manipulate 2D geometric primitives in R2Distance.[cpp/h], R2Segment.[cpp/h], R2Line.[cpp/h], R2Point.[cpp/h], and R2Vector.[cpp/h]. You may find them helpful while writing the morphing functions.
• You can create a zip file to submit with zip -r cos426_assignment1 cos426_assignment1 in the directory containing cos426_assignment1.
• We have found that FFmpeg and MEncoder work well for converting a series of images to a video.
• Post a message to Piazza if you have questions.
• Always remember the late policy and the collaboration policy.

FAQ

Answers to frequently asked questions:

• The compose image parameters don't make sense. Why?
  Unfortunately, the image file formats (bmp, jpg, and ppm) don't store alpha values, so we need a way of getting alpha values for both the top and bottom images. The way we are doing this is by loading a second image for each and treating its blue channel as the alpha value. So... we need 4 images: bottom, bottom_mask, top, and top_mask. bottom is the current image that's already loaded, so the ComposeImage option takes the remaining 3 filenames as input: top, bottom_mask, and top_mask. In main.cpp the 4 images are put together to create two images with the alpha values filled in, which are then passed to the Composite function that you write in R2Image.cpp.

• For the quantization function, should we assume that the number of bits to which we're quantizing is always a multiple of 3?
  No. The parameter is the number of bits per channel, not per pixel.

• I get an assertion error messages and a core dump when the program tries to read in my bmp files. What should I do?
  Make sure your files are 24-bit, uncompressed bitmaps. The provided code reads a specific subtype of .bmp files. You can convert an image to PPM or JPEG format using ImageMagick (this is installed on most of the Princeton Unix machines you can SSH to), or use your favorite paint program to save the file as uncompressed BMP.

• When filtering / sampling / convolving etc, I need pixels outside the image boundary. What do I do?
  There are several ways to deal with that. You can clamp the lookup, that is, you use the closest coordinate that is still in bounds, or you can mirror the look-up on the image boundary. Suppose your image is 10 pixels wide, and you want to retrieve a pixel at x position 12. In clamping mode, you would read the pixel with the highest available coordinate (that is, the one at index 9), when mirroring, you take the pixel at position 7. When implementing error diffusion dithering, you do not need to diffuse errors to pixels outside the image.

• My blur filter is darkening the image. What is wrong?
  Make sure that the entries in your kernel add up to 1. Be especially careful with rounding errors: although small in each entry, collectively they can be large enough to cause this undesired darkening effect.