To See Like a Human: The Quest After Aristotle's Holy Grail
By Taylor Beck
Illustrations by Amey Yun Zhang
A pigeon looks at pictures. The bird knows nothing about heart attacks, nothing about cancer. Yet, trained to classify mammograms and cardiograms, it can outperform doctors.
This ability to sort by sight, without insight, is like the dumb learning done by machines. Online classifiers learn your taste in books, movies, and mates. Similar programs can tell from brain activity what a human or monkey sees- even what he remembers, or dreams. They seem to read minds, and may soon drive cars. Yet their know-how is narrow; they’re like pigeons pecking buttons with mindless competency. As they get smarter, though, these bots are inspiring a new psychology. Just as behaviorists trained pigeons and rats, cognitive scientists are training machines to understand how humans think.
“This is exciting because reasoning about humans in the real world is what motivates us as psychologists,” says Ruairidh Battleday, a graduate student in the department of Computer Science at Princeton. Working with Professor of Psychology and Computer Science Tom Griffiths and postdoctoral researcher Josh Peterson, Battleday has co-authored a series of ambitious papers, proposing a new way to study an ancient riddle.
Philosophers have puzzled over categories since antiquity. Plato proposed ideal types of real-life objects and Linnaeus applied nested categories to classify living species. At the bottom, such concepts are what our minds are made of, the format that structures our thoughts. Yet the classifying we do effortlessly when we see is a challenge to machines. What interests the team at Princeton is how we humans learn the categories that organize our minds.
Years ago, at UC Berkeley, Griffiths learned of a set of images: birds and planes, classified by people (“Is it a bird, is it a plane?”) to train machines. “They had gathered more behavioral data on how people use categories than the entire history of cognitive psychology,” Griffiths says. “We thought, clearly this is a thing that’s possible to do -- to collect rich data sets at this scale.” In a future of self-driving cars and human-free factories, machines will need to discriminate object types quickly. What interested the team, though, was not designing a “perfect” image classifier, like the engineers. They wanted to find a more fallible AI: a machine that “sees like a human.” If a machine makes the same errors we do, if it confuses, say, a blurry deer for a horse-- it could teach us how humans see. Their idea was to train computers not on the “ground truth”-- if that thing in the sky is really a bird, a plane, or Superman -- but on human uncertainty.
The algorithms, called convolutional neural networks, or CNNs, are like “a playground” for psychologists, says Peterson, who co-authored the study. “You have these networks, and there are different knobs you can turn: you can think of any number of hypotheses.” In their case, the hypothesis was that a machine that acts like a person should make the errors a human would. “It’s kind of counter-intuitive,” Peterson adds. “A perfect model-- perfect in the sense that it’s doing the best it can, given the data we’ve given it-- should be confused, because the world is confusing, intrinsically.” Yet engineers of machine vision tend to treat errors as random noise-- not a pattern to be optimized for a more human-like intelligence.
The next step was to peer under the hood of the algorithms to see how they learned. The team did this by pitting four algorithms against one another-- each tuned to a different “setting” for learning. If one algorithm classified natural pictures more the way humans did, it would be considered the “winner”: the most human bot. The team would take note of the knob settings. What representations was the algorithm using, in its hidden layers of structure, to see as we do?
In 2020, the team published a paper in Nature Communications unveiling what they’d found. The paper’s first reference was not an AI pioneer, not Google DeepMind, but Aristotle: Categories, published around 350 B.C.E.. The learning strategies cognitive scientists call “prototype” and “exemplar,” used in this study, echo ancient debates. Is there an ideal Platonic “prototype” for each category, to which each new thing we see is compared? Or are we creatures of flux, as Aristotle had it, our categories continually updated with data?
“I remember surfing with some people a few months ago,” Battleday recalls. “They knew exactly which waves to take. I was there trying to analyze, ‘what would be a good one?’, and I kept getting it wrong.” The surfers navigated the waves, he thinks, because they’d learned a new structure for “what makes a good wave”: a shape of category that doesn’t exist in the amateur’s mind. Battleday struggled to surf, like a pigeon untrained, blinded by the wrong categories.
