COS 426: Computer Graphics Spring 2019

Assignment 1: Imaging (due Sun Feb 24)

Overview

In this assignment you will create a simple image processing program. The operations that you implement will mostly be filters that take an input image (or two), process the image(s), and produce an output image.

Getting Started

The JavaScript based image processing program has two modes: (1) an interactive mode where you can enable/disable various filters, adjust parameters to these filters, and see the result right away; and (2) a batch mode where all the filters and parameters are fixed via the URL string. In general, you will find the interactive mode more useful for testing your programs during development, whereas you will find the batch mode more useful for generating output images while composing the writeups for your assignments.

Same as A0, to get started, download this zip file and unzip it on your computer.

Setting Up a Development Webserver

Next, change to the subdirectory cos426-assign1 and run the command python3 -m http.server in the terminal. (That command is for python 3. In python 2.7 it should be python -m SimpleHTTPServer. On Windows machines, your python 3 executable might just be called python, so you would run your server using python -m http.server. Note that there are many other ways to start a web server, for example using php -S localhost:8000 or by installing WAMP, MAMP, or Node. Various other options are discussed here.) A web server will be launched locally on your computer, rooted at this directory. Note that our assignments will not work if you simply open the html file in the browser directly using file://... due to the same-origin security policy. Therefore we run a local web server.

Once you have started the local web server, direct your browser to http://localhost:8000 (8000 is the default port used by the python web server). If you have done everything correctly up to this point, you should see a picture of a flower, which is the web page for Assignment 1. If you see something else, please reach out to your friends or to an instructor for help.

Before you begin, it is necessary to temporarily disable browser caching, or else changes you make to your code during development may not necessarily be reflected on the web page even after a hard refresh! Many students skim over this step and struggle to get started. For Google Chrome, you can disable cache via the DevTools: first right-click the web page background and choose Inspect Element; then, left-click Network in the toolbar to open the network pane. Finally, check the Disable cache checkbox at the top (you may need to scroll horizontally to find it).

Keep in mind, this setting is only active while the DevTools are open. In general, we strongly recommend Google Chrome for development.

UI Tour

On the right hand side of the web page, you can find two control menus (created using the dat.GUI library). The left panel is the feature menu, which contains a list buttons that, when pressed, apply a filter/tool/operation to the active canvas. Note that it is the goal of the assignment for you to implement most of these features, almost all of which are not implemented in the starter code.

The right panel is the history menu, which displays all operations that are actively applied to the canvas in the order that they were applied.

Please familiarize yourself with the already-implemented features (Push Image, Brightness, Grayscale, Quantize). As an example, we provide the implementation of "Brightness" under the "Luminance" tab. By clicking "Brightness", a new operation will be pushed to the bottom of the history menu, and now you can pull the slider to adjust the brightness of the image. Note that we also provide the implementation of "Grayscale" under the "Color" tab and that of "Quantize" under the "Dithering" tab. If you want to add another image, simply click "Push Image" and an new image will be pushed to the image stack.

You will notice that a warning box pops up when you click on unimplemented features. You will need to comment out or delete the warning box code after you implement the features. All the operations you apply will appear in sequence in the history menu and you can delete the past operations if you want. Observe what happens when you push a number of images onto the history stack, and then click Delete Below on the topmost history element. For other operations, only the selected operation will be removed when Delete is clicked. Also note that all the history operations are encoded in the URL, so feel free to refresh the webpage and everything will still be there. This allows you to change your code and then get back to the same place. It also allows you to change the values directly in the URL rather than in the GUI!

To make the assignment debugging and linking friendly, a "Batch Mode" button is given. Once you are satisfied with the parameters in the interactive GUI, click "Batch Mode" and the batch mode will be loaded in a new tab with all the parameters fixed. Then you can simply refresh when you make changes to your code. For some filters which are slow to process, the batch mode will say "Processing" instead of showing the result immediately. Another cool feature is "Animation". By clicking "Animation", a gif animation will show in the batch mode by varying the value of the most recent ONE-parameter operation. Typically "Animation" is designed for quick operations, so you probably don't want to try this on slow operations such as "Median".

Assignment Features

The assignment is worth 20 points. The following is a list of features that you may implement (listed roughly from easiest to hardest). The number in front of the feature corresponds to how many points the feature is worth for the full implementation. Partial or partially-correct solutions will receive partial credit. The features in bold face are required. The other ones are optional. Refer to the examples web page for example output images as well as some implementation hints.

• Luminance Operations
  • (0.0) Brightness: Change the brightness of an image by scaling RGB values towards black or white.
  • (0.5) Contrast: Change the contrast of an image using the GIMP formula described in Wikipedia.
  • (0.5) Gamma: Apply a gamma correction to the image.
  • (0.5) Vignette: Apply a vignetting operator to the image.
  • (2.0) HistEq: Increase the contrast of the image by histogram equalization in the L channel.
• Color Operations
  • (0.0) Grayscale: Convert to gray levels by replacing each pixel with its luminance.
  • (0.5) Saturation: Change the color saturation of an image using the method described in Graphica Obscura.
  • (1.5) WhiteBalance: Shift all the colors so that a given RGB value becomes white using Von Kries' method.
  • (2.0) HistMatch: Histogram matching - match the L channel to those of another image (easier if you already did HistEq).
• Filter Operations
  • (1.0) Gaussian: Blur an image by convolving it with a Gaussian low-pass filter.
  • (0.5) Sharpen: Apply a linear sharpening filter to the image.
  • (0.5) Edge: Detect edges in an image by convolving it with an edge detection kernel in each color channel.
  • (1.0) Median: Remove speckle noise using a median filter of given width.
  • (2.0) Bilateral: Smooth while preserving sharp edges in the original. See here or here.
• Dithering Operations
  • (0.0) Quantize: Change the number of bits per channel of an image, using simple rounding.
  • (0.5) Random: Perturb each pixel with random noise, and quantize the image using a given number of bits per channel.
  • (1.0) Floyd: Convert an image to a given number of bits per channel, using Floyd-Steinberg error diffusion.
  • (1.0) Ordered: Convert an image to a given number of bits per channel, using a 4x4 ordered dithering matrix.
• Resampling Operations
  • (1.0) Sampling: Implement bilinear and Gaussian sampling (in addition to the basic point sampling). This operation doesn't appear on GUI and should be used as a utility function for your implementations of all other resampling operations and the morph operation below.
  • (0.5) Translate: Translate an image up or down by a number of pixels.
  • (0.5) Scale: Scale an image up or down in resolution by a real valued factor.
  • (1.5) Rotate: Rotate an image by a given angle.
  • (1.5) Swirl: Swirl the image around its center (see the examples web page).
• Composite Operations
  • (1.0) Composite: Push three images into the image stack, paint the third image with black and white using "Brush", use "Get Alpha" to make the third image as the alpha channel of the second image, and blend the second image with the first image using the generated alpha channel (You can also use the provided {'man.jpg', 'doge.jpg', 'alpha.jpg'} as input).
  • (3.0) Morph: Use pairs of corresponding lines to morph a source image into a target image using a morph parameter t. See [Beier92]. Click "MorphLines" to define the line correspondences between the top two images (You can use the provided {trump1.jpg, trump2.jpg, marker.json} as input). Also use "Animation" to generate a gif animation in the batch mode.
• Miscellaneous
  • (3.0) Palette: Create a color palette from an image -- k colors that represent the color scheme of the image -- using basic k-means clustering (aka Lloyd's algorithm). The output should be a new image that contains the original image plus the palette drawn as k rectangles along the right side. See the examples web page.
  • (4.0) Paint: Implement the "paint by numbers" approach described by Haeberli to make the image look like a painting.
  • (4.0) XDoG: Implement the eXtended Difference-of-Gaussians compendium described by Winnemoeller for a range of artistic styles. Hint: use the approach of Kang to make the flow field (instead of the one in the XDoG paper) because it is based on bilateral filter that you already implemented.

Three of the features (brightness, grayscale and quantize) are already implemented for you as examples. By implementing all the required features (in bold), you get 14 points. Full credit for this assignment is 20 points, so to complement the required features you may choose from the optional features listed above and participate in the art contest (which yields one point for participation and two for winning). It is possible to get more than 20 points for this assignment. The value of non-required features incurs diminishing returns after the first 6 points. For sums of non-required features (n>6), each point over 6 will accrue a value of 3/4 the previous point.

Your final score is based on the following factors:

• r: your score on the required features (up to R = 14 points)
• n: your score on the non-required features (value diminishes when n > N = 6) .
• a: your participation in the art contest (1 point for participating, 1.5 for being selected for gallery, 2 for winning). You can make use of customFilter()
• d: diminishing return factor (d = 3/4 in this assignment)

Final score will be rounded to the nearest 0.5. The formula in javascript is:
Math.round((Math.min(R, r) + Math.min(N, n) + d * (1 - Math.pow(d, Math.max(n - N, 0))) / (1 - d) + a) * 2.0) / 2.0

Score Calculator: Compute!

my score on the required features
my score on the non-required features
my score on the art contest
total score

How to Program the Filters

To make your first edit use your favorite text/code editor (we recommend VSCode and Atom) to edit the file js/student.js and fill in your name and NetID. Reload the web page in your browser, and now your information should appear above the image. You should do this for every assignment.

To implement the image processing features listed above, you only need to edit the file js/filters.js. Nevertheless, before starting on that, we recommend you take a quick look at the file js/image.js because it has some important helper code relating to images and pixels. You are welcome to look at any of the other files, but it should not be necessary and some of them have some pretty byzantine JavaScript magic.

In js/filters.js, three of the image processing filters (brightnessFilter, grayscaleFilter and quantizeFilter) are already implemented for you as examples, so you should get the idea of how it works. Some of the filters (like brightness) can work directly in the image that is passed to them; however, other filters (like gaussian) need to allocate a new image to avoid overwriting the original pixel values needed in subsequent processing.

Submitting

You should submit your solution via CS dropbox here. The submitted zip file should preserve the directory structure of the skeleton code we provided in the zip file above. CS dropbox requires that the size of a single file is below 50MB. You can put few largest files in your own web space / google drive / dropbox / youtube (for gif) and include a link to that in the write-up to save space.

The writeup.html file should be an HTML document demonstrating the effects of the features you have implemented and would like scored. For the features you would like to demonstrate, make sure that you include the required results by replacing placeholder.png(s) with you results. You are encouraged to include more representative results, but extra results only affect your score when your implementation is partially correct. You don't have to show the input images for the required results. Additionally, please write down who you have collaborated with for this assignment in the last "collaboration" section of writeup.html.

You should start from the the example writeup.html provided. At the top of that file are a list of features that you might implement, linking to the section where you talk about them. Please remove any features that you do not implement from the list as well as the corresponding sections, but otherwise leave this header section in tact. When you include an extra result, also include a link to the batch.html command that creates that image, as illustrated in the "Brightness" example.

To save space, please submit images in png format. You may include one or a few gif files as well to show animations, but these files can be large so please try not to include lots of large gifs.

Note that you are expected to use good programming style at all times, including meaningful variable names, a comment or three describing what the code is doing, etc. Partial credit may not be assigned for code without comments. We have mostly tried to conform to the idiomatic JS style conventions.

A Note on Grading

Your assignment submissions for this course will be graded by manual inspection, not by scripts. As such, do not stress about "pixel-perfect" accuracy. For instance, if your gaussian filter implementation uses a different padding strategy at the image boundary from our reference example, you will still receive full-credit! We care far more about your understanding, and likely won't even notice small discrepencies so long as your result is generally correct.

Debugging Tips

In general, graphics programs are very challenging to debug. Throughout this course, print statements via the console.log() function will prove invaluable. Additionally, if you are using Chrome, there is a built-in JavaScript debugger under the sources pane that will allow you to both set execution breakpoints and trace variables. If you are using VSCode or Atom, there are also packages that allow the editors to interface directly into the Chrome debugger/console.

Hints

A few hints:

• Do the simplest filters first!
• Look at the example page.
• Look at Week 2 and Week 3's precept slides, which cover the image processing features in details.
• Post a message to Piazza if you have questions.
• Take note of the late policy and the collaboration policy.

FAQ

Here are some answers to frequently asked questions. Check back here occasionally, as we may add FAQs to this list:

• How do I add my own images or .json files?
  The file lists are hardcoded in coursejs/guiConfig.js because javascript does not have access to the filesystem. Please modify this file when needed.

• When filtering / sampling / convolving etc, I need pixels outside the image boundary. What do I do?
  There are several ways to deal with that. You can clamp the lookup, that is, you use the closest coordinate that is still in bounds. Suppose your image is 10 pixels wide, and you want to retrieve a pixel at x position 12. In clamping mode, you would read the pixel with the highest available coordinate (that is, the one at index 9). You can also omit pixels outside the boundary from a convolution, but make sure that the remaining weights sum to 1 (for kernels that need this property). When implementing error diffusion dithering, you do not need to diffuse errors to pixels outside the image.

• My blur filter is darkening the image. What is wrong?
  Make sure that the entries in your kernel add up to 1. Be especially careful with rounding errors: although small in each entry, collectively they can be large enough to cause this undesired darkening effect.