COS 526: Advanced Computer Graphics Fall 2008

You may also use any other source code as reference material, and also as library source, provided it doesn't directly solve the assigned problems. When you use other people's sources, please reference it in your writeup.

You program should run from the command line, taking one or more image file names and a set of parameters on the command line. It should read and write the files based on those parameters. You may create input examples any way you choose, using Photoshop or any other image-editing software (especially for mattes), grabbing them from the web or wherever.

Part 1: Texture Synthesis

1. Basic method: Synthesize larger examples from smaller examples. Replicate the examples shown here. Also try some of the different kinds of textures shown here. You can also find many examples on the web.
2. Constrained synthesis: Extrapolate from a smaller rectangular image to fill a border area or a hole, as shown at the bottom of the page here. Try using the technique for in-painting to remove some "scratches" you put in a few photographs (by drawing on them in a drawing program, and creating a matte that identifies the scratches).
3. Optional: Try to accelerate this very slow method somehow. Some ideas: When finding good matches, only compare against a small random subset of all patches; scan pixels in order and remember the best matches from the previous pixel, while also considering a some new randomly chosen locations. Or, try to accelerate the search for a good match by some kind of high-dimensional neighborhood search structure, such as K-D trees or vector quantization.

Part 2: Seam Carving

1. Basic method: make a command line tool that reads in a file and writes out a series of files that change the aspect ratio. The command line parameters should specify how the image shape should change and how many frames to write out. Only consider cases that make the image smaller in one dimension and keep the other dimension fixed. Suggestion: only handle on "vertical" cuts; to handle horizontal cuts rotate the image before (and after) by 90 degrees.
2. Optional stretching: also allow the image to become larger.
3. Optional arbitrary size: allow the image to change in both dimensions.
4. Optional protection: also incorporate a matte that specifies regions that should not contain seams (e.g. faces).
5. Optional movie: use any video sofware (e.g. ffmpeg or mencoder) to take the output sequence turn it into a short video clip.
6. Optional fancy interface: if you want to get fancy with the interface, make a GUI that allows you to drag the window borders.

Part 3: Texture synthesis using image quilting

1. Warm up "chaos mosaic": implement the method of Xu et al (PDF). This method simply pastes rectangles randomly from the source into the target. Do not bother with the specific random patterns they describe -- just use uniform random patch placement, ideally with a little blending at the edges.
2. Basic image quilting: Implement the image quilting texture synthesis scheme of Efros and Freeman. This uses dynamic programming to do the minimal-error bounding cut at the boundaries. There are some examples of source code available on the web, for example here and here.
3. Optional: do something like Efros and Freeman's texture transfer.

Part 4: (optional) Art Contest