Editor’s Note (10/5/18): From a relatively young age, Doris Tsao distinguished herself as a leading scientist investigating the way the brain recognizes faces. On October 4 the John D. and Catherine T. MacArthur Foundation recognized Tsao by giving her one its 25 fellowships (“genius” awards) for 2018 ($625,000 allocated over the next five years). In doing so, the foundation acknowledged: “her most notable line of research has focused on uncovering the fundamental neural principles that underlie one of the brain’s most highly specialized and socially important tasks—recognizing a face.” A news item published last year on Tsao's work on the neural code for faces follows. Stay tuned in coming months for an article by Tsao on face recognition in Scientific American.
The brain has evolved to recognize and remember many different faces. We can instantly identify a friend’s countenance among dozens in a crowded restaurant or on a busy street. And a brief glance tells us whether that person is excited or angry, happy or sad.
Brain-imaging studies have revealed that several blueberry-size regions in the temporal lobe—the area under the temple—specialize in responding to faces. Neuroscientists call these areas “face patches.” But neither brain scans nor clinical studies of patients with implanted electrodes explained exactly how the cells in these patches work.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Now, using a combination of brain imaging and single-neuron recording in macaques, biologist Doris Tsao and her colleagues at the California Institute of Technology appear to have finally cracked the neural code for primate face recognition. The researchers found the firing rate of each face patch cell corresponds to a separate facial feature. Like a set of dials, the cells can be fine-tuned to respond to bits of information, which they can then combine in various ways to create an image of every face the animal encounters. “This was mind-blowing,” Tsao says. “The values of each dial are so predictable that we can re-create the face that a monkey sees by simply tracking the electrical activity of its face cells.”
Previous studies had hinted at the specificity of these brain areas for encoding faces. In the early 2000s, when Tsao was a postdoctoral researcher at Harvard Medical School, she and electrophysiologist Winrich Freiwald showed that neurons in a monkey’s face patches would fire electrical signals every time the animal saw pictures of a face. But the same brain cells showed little or no response to other objects, such as images of vegetables, radios or nonfacial body parts. Other experiments indicated that neurons in these regions could also distinguish among individual faces, even if they were cartoons.
In a famous set of experiments in human subjects in 2005, neuroscientist Rodrigo Quian Quiroga found that pictures of actor Jennifer Aniston activated a single brain cell in the hippocampus region—the so-called Jennifer Aniston neuron. A similar process was thought to occur elsewhere in the temporal lobe, where the prevailing theory held that each neuron in the face patches was sensitive to a few particular people, says Quian Quiroga, who is now at the University of Leicester in England and was not involved with the current work. But Tsao’s recent study suggests that theory may be mistaken. “She has shown that neurons in face patches don’t encode particular people at all; they just encode certain features,” Quian Quiroga says. “That completely changes our understanding of how we recognize faces.”
To decipher how cells perform this recognition task, Tsao and postdoc Steven Le Chang generated 2,000 human mug shots with variations in 50 features, including facial roundness, distance between the eyes, and skin tone and texture. They showed these images to two monkeys while recording electrical activity from individual neurons in three separate face patches in both animals.
Each neuron responded to only a single feature, the researchers found. Rather than encoding individuals’ faces, like the Jennifer Aniston neuron in the hippocampus, the face patch neurons were dividing images into smaller regions and encoding specific features such as hairline width, Chang says. Moreover, the neurons in separate face patches processed complementary information. Like factory workers, the various face patches had distinct jobs, cooperating, communicating and building on one another to provide a complete picture.
Once Chang and Tsao knew how this division of labor occurred, they could predict the neurons’ responses to a completely novel face. They developed a mathematical model in which facial features were encoded by various neurons. Then they showed monkeys a previously unseen image of a human face (1). Using their algorithm for how various neurons would respond, the researchers were able to digitally re-create the visage that a monkey had viewed (2). “The re-creations were stunningly accurate,” Tsao says. In fact, they were nearly indistinguishable from the actual pictures the monkeys saw.
The original face (1) presented to a monkey and the face predicted by its brain activity (2). Credit: Courtesy of Steven Le Chang California Institute of Technology
Even more surprisingly, the researchers needed readings from only a relatively small set of neurons for the algorithm to accurately re-create the faces monkeys were seeing, Tsao says. Recordings from just 205 cells—106 in one patch and 99 in another—were enough. “It really speaks to how compact and efficient this feature-based neural code is,” she says. It may also explain why primates are so good at facial recognition and how we can potentially distinguish among billions of different people without needing an equally massive number of face cells.
The findings, which were published recently in Cell, provide scientists with a comprehensive, systematic model for how the brain perceives faces. This human cerebral machinery is very similar to that of monkeys, and we have face patches that respond like theirs to images in functional MRI studies, according to researchers. Yet the number of human face patches might differ.
Understanding the brain’s facial code could help scientists study how face cells incorporate other identifying information, such as sex, age, race, emotional cues and names, says Adrian Nestor, a neuroscientist at the University of Toronto, who studies face patches in human subjects and did not participate in the research. It may even provide a framework for decoding how the brain processes nonfacial shapes. “Ultimately this puzzle is not just about faces,” he explains. “The hope is that this neural code extends to object recognition as a whole.”