Spring 2017

Assignment 1: Image Processing

Overview

In this assignment you will create a simple image processing program. The operations that you implement will mostly be filters that take an input image (or two), process the image(s), and produce an output image.

Getting Started

The JavaScript based image processing program has two modes: (1) an interactive mode where you can enable/disable various filters, adjust parameters to these filters, and see the result right away; and (2) a batch mode where all the filters and parameters are fixed. Each version runs in a web page, and for the batch mode the filters and parameters are specified in the URL.

To get started, download this zip file and unzip it on your computer. Change to the subdirectory cos426-assign1 and run the command python -m SimpleHTTPServer in the terminal. (That command is for python 2.7. In python 3 it should be python3 -m http.server. There are also many other ways to start a web server, for example using php -S localhost:8000 or by installing WAMP or MAMP or Node. Various other options are discussed here.) A web server will be launched locally on your computer, rooted at this directory. Note that our assignments will not work if you simply open the html file in the browser directly using file://... due to the same-origin security policy. Therefore we run a local web server.

Once you have started the local web server, direct your browser to http://localhost:8000 (8000 is the default port used by the python web server). You should see the web page for Assignment 1 showing a picture of a flower. On the right hand side of the web page, you can find two control menus. The left one is the feature menu which contains the features/operations you will need to implement in this assignment and the right one is the history menu which keeps track of the history operations. In the feature menu, you can expand the folders and see all the features. As an example, we provide the implementation of "Brightness" under the "Luminance" folder. By clicking "Brightness", a new operation will be pushed to the bottom of the history menu, and now you can pull the slider to adjust the brightness of the image. Note that we also provide the implementation of "Grayscale" under the "Color" folder. If you want to add another image, simply click "Push Image" and an new image will be pushed to the image stack.

Please familiarize yourself with the already-implemented features (Push Image, Brightness, Grayscale, Fill). For those features which have not been implemented yet, a warning box will pop up if you click on them. You will need to comment out the warning box code after you implement the features. All the operations you apply will appear in sequence in the history menu and you can delete the past operations if you want. For the "Push Image" operation, by clicking "Delete Below", all the operations below it will be removed as well because those operations are associated with the image being removed. For other operations, by clicking "Delete", only the selected operation will be removed. Also note that all the history operations are encoded in the URL, so feel free to refresh the webpage and everything will still be there. This allows you to change your code and then get back to the same place. It also allows you to change the values directly in the URL rather than in the GUI.

To make debugging friendly, a batchMode button is given. Once you are satisfied with the parameters in the interactive GUI, click "BatchMode" and the batch mode will be loaded in a new tab with all the parameters fixed. Then you can simply refresh when you make changes to your code. For some filters which are slow to process, the batch mode will say "Processing" instead of showing the result immediately. Another cool feature is "Animation". By clicking "Animation", a gif animation will show in the batch mode by varying the value of the most recent ONE-parameter operation. Typically "Animation" is designed for quick operations, so you probably don't want to try this on slow operations such as "Median".

Use your favorite text/code editor to edit the file js/student.js and fill in your name and NetID. Reload the web page in your browser, and now your information should appear above the image.

Implementing the Image Processing Filters

The assignment is worth 20 points. The following is a list of features that you may implement (listed roughly from easiest to hardest). The number in front of the feature corresponds to how many points the feature is worth for the full implementation. Partial or partially-correct solutions will receive partial credit. The features in bold face are required. The other ones are optional. Refer to the examples web page for example output images as well as some implementation hints.

• Set Pixels Operations
  • (0.0) Fill: Fill the whole image with a given color.
  • (0.5) Brush: Implement a color brush which takes color, mouse coordinates and brush radius as inputs.
• Luminance Operations
  • (0.0) Brightness: Change the brightness of an image by scaling RGB values towards black or white.
  • (0.5) Contrast: Change the contrast of an image using the GIMP formula described in Wikipedia.
  • (0.5) Gamma: Apply a gamma correction to the image.
  • (0.5) Vignette: Apply a vignetting operator to the image.
  • (2.0) HistEq: Increase the contrast of the image by histogram equalization in the L channel.
• Color Operations
  • (0.0) Grayscale: Convert to gray levels by replacing each pixel with its luminance.
  • (0.5) Saturation: Change the color saturation of an image using the method described in Graphica Obscura.
  • (1.5) WhiteBalance: Shift all the colors so that a given RGB value becomes white using Von Kries' method.
  • (2.0) HistMatch: Histogram matching - match R,G,B channels to those of another image (easier if you already did HistEq).
• Filter Operations
  • (1.0) Gaussian: Blur an image by convolving it with a Gaussian low-pass filter.
  • (0.5) Sharpen: Apply a linear sharpening filter to the image.
  • (0.5) Edge: Detect edges in an image by convolving it with an edge detection kernel in each color channel.
  • (1.0) Median: Remove speckle noise using a median filter of given width.
  • (2.0) Bilateral: Smooth while preserving sharp edges in the original. See here or here.
• Resampling Operations
  • (1.0) Sampling: Implement bilinear and Gaussian sampling (in addition to the basic point sampling) for all resampling operations, including the morph below.
  • (0.5) Translate: Translate an image up or down by a number of pixels.
  • (0.5) Scale: Scale an image up or down in resolution by a real valued factor.
  • (1.5) Rotate: Rotate an image by a given angle.
  • (1.5) Swirl: Swirl the image around its center (see the examples web page).
• Composite Operations
  • (1.0) Composite: Push three images into the image stack, paint the third image with black and white using "Brush", use "Get Alpha" to make the third image as the alpha channel of the second image, and blend the second image with the first image using the generated alpha channel (You can also use the provided {'man.jpg', 'doge.jpg', 'alpha.jpg'} as input).
  • (3.0) Morph: Use pairs of corresponding lines to morph a source image into a target image using a morph parameter t. See [Beier92]. Click "MorphLines" to define the line correspondences between the top two images (You can use the provided {chang.jpg, halber.jpg, marker.json} as input). Also use "Animation" to generate a gif animation in the batch mode.
• Miscellaneous
  • (3.0) Palette: Create a color palette from an image -- k colors that represent the color scheme of the image -- using basic k-means clustering (aka Lloyd's algorithm). The output should be a new image that contains the original image plus the palette drawn as k rectangles along the right side. See the examples web page.
  • (4.0) Paint: Implement the "paint by numbers" approach described by Haeberli to make the image look like a painting.
  • (4.0) XDoG: Implement the eXtended Difference-of-Gaussians compendium described by Winnemoeller for a range of artistic styles. Hint: use the approach of Kang to make the flow field (instead of the one in the XDoG paper) because it is based on bilateral filter that you already implemented.

A few of the features (fill, brightness, grayscale) are already implemented for you as examples. By implementing all the required features (in bold), you get 13 points. Full credit for this assignment is 20 points, so to complement the required features you may choose from the optional features listed above and submit your work to the art gallery (one point - you can make use of customFilter()). Your final score will be calculated by adding:

• your score on the required features (up to 13 points)
• your score on the non-required features and submitting to the art gallery, calculated as follows.

The first 7 points worth of non-required features simply add to your score. Beyond 7, each additional point of non-required features counts 2/3 as much as the previous point. (For example, if you implement 9 points of non-required features, this will count as 7 + 2/3 + 4/9 = 8.1 points.) The maximum score on this assignment is 23/30. (Yay infinite series!)

How to Program the Filters

To implement the image processing features listed above, you only need to edit the file js/filters.js. Before starting on that, we recommend you take a quick look at the file js/image.js because it has some important helper code relating to images and pixels. You are welcome to look at any of the other files, but it should not be necessary and some of them have some pretty byzantine javascript magic. In js/filters.js, two of the image processing filters are already implemented for you as examples, so you should get the idea of how it works. Some of the filters (like brightness) can work directly in the image that is passed to them; however, other filters (like gaussian) need to allocate a new image to avoid overwriting the original pixel values needed in subsequent processing.

Submitting

The Dropbox link to submit your assignment is here.

You should submit a single zip file named yournetid.zip. The submitted zip file should preserve the directory structure of the skeleton code we provided in the zip file above. CS dropbox requires the size of a single file to be below 50MB. If necessary, you can put few of the largest files in your own web space / google drive / dropbox and include a link to that in the write-up to save space.

The writeup.html file should be an HTML document demonstrating the effects of the features you have implemented and would like scored. For the features you would like to demonstrate, make sure that you include the required results by replacing placeholder.png(s) with you results. You are encouraged to include more representative results, but extra results only affect your score when your implementation is partially correct. You don't have to show the input images for the required results.

You should start from the the example writeup.html provided. At the top of that file are a list of features that you might implement, linking to the section where you talk about them. Please remove any features that you do not implement from the list as well as the corresponding sections, but otherwise leave this header section in tact. When you include an extra result, also include a link to the batch.html command that creates that image, as illustrated in the "Brightness" example.

To save space, please submit images in png format. You may include one or a few gif files as well to show animations, but these files can be large so please try not to include lots of large gifs.

Note that you are expected to use good programming style at all times, including meaningful variable names, a comment or three describing what the code is doing, etc. Partial credit may not be assigned for code without comments. We have mostly tried to conform to the idiomatic JS style conventions.

Hints

A few hints:

• Do the simplest filters first!
• Look at the example page.
• Post a message to Piazza if you have questions.
• Take note of the late policy and collaboration policy posted on the main assignments page.

FAQ

Here are some answers to frequently asked questions. Check back here occasionally, as we may add FAQs to this list:

• How do I add my own images or .json files?
  The file lists are hardcoded in js/guiConfig.js because javascript does not have access to the filesystem. Please modify this file when needed.

• When filtering / sampling / convolving etc, I need pixels outside the image boundary. What do I do?
  There are several ways to deal with that. You can clamp the lookup, that is, you use the closest coordinate that is still in bounds. Suppose your image is 10 pixels wide, and you want to retrieve a pixel at x position 12. In clamping mode, you would read the pixel with the highest available coordinate (that is, the one at index 9). You can also omit pixels outside the boundary from a convolution, but make sure that the remaining weights sum to 1 (for kernels that need this property). When implementing error diffusion dithering, you do not need to diffuse errors to pixels outside the image.

• My blur filter is darkening the image. What is wrong?
  Make sure that the entries in your kernel add up to 1. Be especially careful with rounding errors: although small in each entry, collectively they can be large enough to cause this undesired darkening effect.