Meta-unsupervised-learning: a principled approach to unsupervised learning

Unsupervised Learning and exploratory data analysis are among the most important and yet murkiest areas within machine learning. Debates rage even about how to choose which objective function to optimize. We introduce a principled data-driven approach: “meta-unsupervised-learning” using a collection of related or unrelated learning problems. We present simple agnostic models and algorithms illustrating how the meta approach circumvents impossibility results for novel "meta" problems such as meta-clustering, meta-outlier-removal, meta-feature-selection, and meta-embedding. We also present empirical results showing how the meta approach improves over standard techniques for problems such as outlier removal and choosing a clustering algorithm and a number of clusters. We also train an unsupervised neural network that learns from prior supervised classification problems drawn from learning problems at openml.org.

Joint work with Vikas Garg from MIT

Adam Tauman Kalai received his BA from Harvard, and MA and PhD under the supervision of Avrim Blum from CMU. After an NSF postdoctoral fellowship at M.I.T. with Santosh Vempala, he served as an assistant professor at the Toyota Technological institute at Chicago and then at Georgia Tech. He is now a Principal Researcher at Microsoft Research New England. His honors include an NSF CAREER award and an Alfred P. Sloan fellowship. His research focuses on machine learning, human computation, and algorithms.