SHAPE ANALYSIS WITH CROWDSOURCED DATA

Abstract:

With explosion of 3D geometric data and its increasing usage in many applications, ranging from computer-aided design to medicine to paleontology, shape analysis is becoming an important research field. Common to many shape analysis tasks are two sub-problems: 1) segmenting a shape into meaningful parts and 2) identifying important/salient points on a shape. Both are easier for humans than for computers. In this thesis, we revisit these problems, from the angle of using crowdsourced data to learn from humans. We first investigate the problem of mesh segmentation. By constructing a benchmark of 4300 manually generated segmentations for 380 surface meshes of 19 different object categories, and developing software to analyze 11 geometric properties of segmentations and to compute 4 quantitative metrics for comparison of segmentations, we are able to quantitatively answer “How do people decompose shapes into meaningful parts” and “How algorithms do in comparison with humans”. The benchmark is widely adopted for evaluating new segmentation algorithms and prompts emergence of learning-based algorithms for mesh segmentation and labeling. We then visit the problem of identifying salient points on meshes. Rather than defining saliency bottom-up from low-level geometric features, we start with the social/psychological essence of saliency by investigating Schelling points [1] on 3D meshes. We designed an online pure coordination game that asked people to select points on 3D surfaces that they expect will be selected by other people. We then analyzed properties of the selected points, finding that Schelling point sets are usually highly symmetric, that local curvature properties proposed in previous work are most helpful for identifying obvious Schelling points while global properties (e.g., segment centeredness, proximity to symmetry axis, etc.) are required to explain more subtle features. Based on these observations, we use regression to combine multiple properties into an analytical model that predicts where Schelling points are likely to be on new meshes. iii hared by these two problems are the desire to transfer properties and distributions collected for known meshes to unseen ones. Finally, we propose MeshMatch, a mesh-based analogy of the image-based PatchMatch [2] algorithm and a non-parametric multi-resolution approach to surface property transfer based on this algorithm. This method offers benefits of both parametrization and texture synthesis approaches to preserving both large- and finescale patterns of the source properties. We show its applications in texture transfer, detail transfer, texture painting, and transferring Schelling distributions.