Exploiting compositionality to explore a large space of model structures

I will present flexible algorithms for model discovery and model fitting which apply to broad, open-ended classes of models, yet which also incorporate model-specific algorithmic insights. First, I will introduce a framework for building probabilistic models compositionally out of common modeling motifs, such as clustering, sparsity, and dimensionality reduction. This compositional framework yields a variety of existing models as special cases. We can flexibly perform posterior inference across this large, open-ended space of models by composing sophisticated inference algorithms carefully designed for the individual modeling motifs. An automatic structure search procedure over this space of models yields sensible analyses of datasets as diverse as motion capture, natural image patches, and Senate voting records, all using a single software package with no hand-tuned metaparameters. Applying a similar compositional structure search procedure to Gaussian Process models yields interpretable decompositions of diverse time series datasets and enables automatic generation of natural language reports.  Finally, compositional structure search depends crucially on the estimation of intractable likelihoods. I will briefly outline an approach for obtaining precise likelihood estimates with rigorous tail bounds by sandwiching the true value between stochastic upper and lower bounds.

Roger Grosse is a Postdoctoral Fellow in the University of Toronto machine learning group. He received his Ph.D. in computer science from MIT under the supervision of of Bill Freeman. He is a recipient of the NDSEG Graduate Fellowship, the Banting Postdoctoral Fellowship, and outstanding paper awards at the International Conference of Machine Learning (ICML) and the Conference for Uncertainty in AI (UAI). He is also a co-creator of Metacademy, an open-source web site for developing personalized learning plans in machine learning and related fields.