image: iStock
Why do our eyes tend to be drawn to certain shapes, colors and silhouettes more than others?
For more than half a century, researchers have known that neurons in the brain’s visual system respond more to some images than others—a feature that is critical for the ability to recognize, understand and interpret the multitude of visual clues surrounding us.
For example, specific populations of visual neurons in an area of the brain known as the inferior temporal cortex fire more when people or other primates—animals with highly attuned and visual systems—look at faces, places, objects or text. But exactly what these neurons are responding to has remained unclear.
Now a small study in macaques led by investigators in the Blavatnik Institute at Harvard Medical School has generated some valuable clues based on an artificial intelligence system that can reliably determine what neurons in the brain’s visual cortex prefer to see.
A report of the team’s work is published May 2 in Cell.
The vast majority of experiments to date that attempt to measure neuronal preferences have used real images. But real images carry an inherent bias: They are limited to stimuli available in the real world and to the images that researchers choose to test. The AI-based program overcomes this hurdle by creating synthetic images tailored to the preference of each neuron.
Will Xiao, a graduate student at Harvard University, designed a computer program that uses a form of responsive artificial intelligence to create self-adjusting images based on neural responses obtained from six macaque monkeys. To do so, the researchers measured the firing rates from individual visual neurons in the brains of the animals as they were watching images on a computer screen.
Over the course of a few hours, the animals were shown images in 100 millisecond blips, each generated by Xiao’s program. The images started out with a random textural pattern in grayscale. Based on how much the monitored neurons fired, the program gradually introduced shapes and colors, morphing over time into a final image that fully embodied a neuron’s preference. Because each of these images is synthetic, Xiao said, it avoids the bias that researchers have traditionally introduced by only using natural images.
“At the end of each experiment,” he said, “this program generates a super stimulus for these cells.”
The results of these experiments were consistent over separate runs, explained study senior investigator Margaret Livingstone, the Takeda Professor of Neurobiology at HMS. Specific neurons tended to evolve images through the program that weren’t identical but were remarkably similar.
Some of these images were in line with what she and her colleagues expected. For example, a neuron that they suspected might respond to faces evolved round pink images with two big black dots akin to eyes. Others were more surprising. For example, a neuron in one of the animals consistently generated images that looked like the body of a monkey, but with a red splotch near its neck. The researchers eventually realized that this monkey was housed near another that always wore a red collar.
“We think this neuron responded preferentially not just to monkey bodies but to a specific monkey,” said Livingstone.
Not every final image looked like something recognizable, Xiao added. One monkey’s neuron evolved a small black square. Another evolved an amorphous black shape with orange below.
Livingstone notes that research from her lab and others has shown that the responses of these neurons are not innate—instead, they are learned through consistent exposure over time to visual stimuli. When this ability to recognize and fire preferentially to certain images arises is unknown, Livingstone said. She and her colleagues plan to investigate this question in future studies.
Learning how the visual system responds to images could be key to better understanding the basic mechanisms that drive cognitive issues ranging from learning disabilities to autism spectrum disorders, which are often marked by impairments in a child’s ability to process facial cues and to recognize faces.
“This malfunction in the visual processing apparatus of the brain can interfere with a child’s ability to connect, communicate and interpret basic cues,” she said. “By studying those cells that respond preferentially to faces, for example, we could uncover clues to how social development takes place and what might sometimes go awry.”
The research was funded by National Institutes of Health (grants R01EY16187, R01EY25670, R01EY011379, P30EY012196 and R01EY026025) and National Science Foundation award CCF-1231216.
