Tools for the artist in all of us
FROM TRANSLATING FOREIGN LANGUAGES to finding information in minutes, computers have extended our productivity and capability. But can they make us better artists?
Researchers in the Department of Computer Science are working on ways to make it easier to express artistic creativity without the painstaking hours spent learning new techniques. “Computers are making it faster and easier for beginners to do a lot of things that are time-consuming,” said Jingwan (Cynthia) Lu, who earned her Ph.D. at Princeton in spring 2014. “I’m interested in using computers to handle some of the more tedious tasks involved in the creation of art so that humans can focus their talents on the creative process.”
The techniques that Lu is creating are far more versatile than the simple drawing and painting tools that come pre-installed on most computers, yet they are much easier to use than the software marketed to artists and designers. “Lu has created tools that enable artistic expression by leveraging the use of computation,” said Professor of Computer Science Adam Finkelstein, Lu’s dissertation adviser.
Last year, Lu introduced RealBrush, a project that permits people to paint on a computer using a variety of media, ranging from traditional paints to unconventional materials such as glittered lip gloss. The software contained a library of photographs of real paint strokes. As the artist painted on a tablet or touch screen, the software pieced together the stored paint strokes.
This year, Lu has introduced two new techniques that further her goal of making it easy to create art digitally:
decoBrush
decoBrush allows the user to create floral designs and other patterns such as those found as borders on invitations and greeting cards. Many design programs offer such borders but they come in set shapes and are not easy to customize, requiring a designer to painstakingly manipulate individual curves and shapes.
With decoBrush, users can create highly structured patterns simply by choosing a style from a gallery and then sketching curves to form the intended design or layout. The decoBrush software transforms the sketched paths into structured patterns in the style chosen. For example, a user might select a floral pattern and then sketch a heart, creating a heart with a floral border.
The challenge for Lu and her codevelopers was to guide the computer to learn existing decorative structured patterns and then apply automatic algorithms to replace the tedious process of manipulating the individual curves and shapes.
“Given a target path such as a sketch that the pattern should follow, the computer copies, alters and merges segments of existing pre-designed patterns, which we call ‘exemplars,’ to compose a new pattern,” Lu said. “It does this by searching for candidate segments that have similar curviness to the target sketch that the user drew. The candidate segments are then copied and merged using a specialized texture synthesis algorithm that transforms the curves to align with each other seamlessly at the segment boundaries.”
Lu constructed decoBrush with assistance from Connelly Barnes, who earned his doctorate degree from Princeton in 2011 and is now at the University of Virginia; undergraduate Connie Wan, Class of 2014; and Finkelstein. She also collaborated with Paul Asente and Radomir Mech of Adobe Research, where Lu interned for three summers and now works as a researcher. Lu presented decoBrush at the Association of Computer Machinery Siggraph Conference in August 2014.
A second project enables artists and novices to explore mixing of colors in digital painting, with the goal of making the digital results more faithful to the physical behaviors of paints.
Software programs for painting are not adept at combining colors, especially when they are simulating complex media such as oil paints or watercolors. One of the most common techniques for combining colors, alpha blending, estimates that yellow and blue make gray rather than green. Lu and her colleagues came up with a different method for figuring out how colors will blend using techniques borrowed from real-world (non-digital) painting.
The researchers use color charts that artists make to find out what color arises when overlaying or mixing two colors of paint. Making these color charts involves painting rows of one color, and then overlaying them with columns each containing a different color. The resulting grid reveals how all pairs of color will look when layered. Similar charts can be made for mixed rather than layered colors.
Lu’s approach is to feed these color charts into the computer to teach it how to combine colors in a specific medium, such as oil paints or watercolors. “The goal is to learn from existing charts to predict the result of compositing new colors,” Lu said. “We apply simplifying assumptions and prior knowledge about pigment properties to reduce the number of learning parameters, which allows us to perform accurate predictions with limited training data.”
Lu’s research was supported by a Siebel Fellowship and funding from Google. The project included Willa Chen, Class of 2013; Stephen DiVerdi of Google; Barnes and Finkelstein. The work was presented at the June 2014 International Symposium on Non-Photorealistic Animation and Rendering.
https://discovery.princeton.edu/2014/11/17/computer-science-tools-for-the-artist-in-all-of-us/
