The colliding of one plane and then another into the World Trade Center towers on a brilliant late summer day in 2001 was the ultimate in surprise endings. Many wondered what was going on in the minds of the men who commandeered the planes, but what about us? What was going on in our minds, and in our brains, as we watched all our expectations about what passenger planes typically do be violated?
Gina Kuperberg, Tatiana Sitnikova, and their colleagues have been conducting experiments that could help answer this question. Over the past eight years, they have been exposing subjects to all manner of verbal and visual surprise endings—sentences with unexpected or outlandish words and video clips with anomalous or downright bizarre final images. Using a combination of methods for detecting when and where neural activity occurs, they have been comparing how our brains react to the merely unexpected scenarios versus the wildly strange ones.
Their findings, published over the last few years, add up to their own version of a surprise twist. Though both types of anomalies are processed rapidly, in under a second, the more outlandish ones take a bit longer. Though the delay is slight, a mere 200 milliseconds, the lag—and also the accompanying brain activity—look more like what occurs when we try to comprehend grammatical mistakes rather than errors of meaning.
Now Kuperberg and Sitnikova, both at Massachusetts General Hospital, have used this difference in timing and activity to develop a new model for how we make sense of events, both verbally and visually. In their model, presented by Kuperberg in an upcoming special issue of Brain Research, comprehension occurs along two separate though interacting neural streams. The first, and faster, comes into play as the brain attempts to map new input to what it already has seen or experienced. When this initial, more rigid memory-based system fails—because the perceived actions appear too unfamiliar or nonsensical—another wave of brain activity occurs. This may reflect the workings of a second system, one that compares the relationship between subject and action to an implicit set of rules to see if the action is something a particular object or person could, in principle, perform.
“At that moment when we saw the airplanes crash on 9/11, our brains, I predict, would have really been engaging this network,” said Kuperberg, associate psychiatrist at MGH and HMS lecturer on psychiatry. Sitnikova is an HMS research fellow in neurology.
“We knew immediately, intuitively, and tragically that the planes were basically being used as human missiles, in a way that has never been done before,” Kuperberg said.
The researchers’ work suggests that the two streams—memory-based and action-based—are called into play not just by short sentences or video clips but as we perceive longer sequences of events, such as those conveyed in stories and movies. And they may be provoked by ordinary events, not just the unexpected or outrageous.
“I think the system is there in the first place—we’re probing it through violations, but we use it and need it all the time,” said Kuperberg. “The first system makes use of prior real-world knowledge stored in memory to guide everyday comprehension, to prepare us for stimuli that are likely to come next. This matching only goes so far, and then the brain must call into play a more flexible route.” By cultivating this more flexible route, we might better prepare ourselves for extraordinary events, like 9/11.
A key feature of the model is that the two systems exist in balance. This equilibrium, Kuperberg and Sitnikova argue, is what enables us to deal with a world that is both familiar and novel, even wildly so. Yet it may be thrown off in people with schizophrenia and other psychiatric disorders. People with schizophrenia exhibit disordered thought—they spin associations between individual words at the expense of building up meaning as a whole. They also are prone to delusions.
“I think the reprocessing network is basically disrupted in people with schizophrenia and that leaves them more dependent on individual associations and prior experience,” said Kuperberg. An overactive memory-based system could cause them to formulate nonexistent associations, leading to disordered thoughts. In conjunction with abnormal emotional processing, another of schizophrenia’s hallmarks, it could cause them to jump to conclusions about input that is normally reprocessed by the action-based system, resulting in delusions.
Trained as a psychiatrist, Kuperberg was captivated by the bizarre patterns of thought and speech of her schizophrenic patients. “I wanted to understand why the phenomena they were exhibiting happened,” she said. It was the 1990s and linguistics and cognitive science were in thrall to the classic distinction between syntax and semantics. In fact, the distinction had been reified by two recent discoveries.
In 1980, Marta Kutas of the University of California, San Diego, found that subjects exhibited an event-related potential (ERP), a standard measure of brain activity, 400 milliseconds after hearing a semantic violation such as “She spread the bread with butter and socks.” The change was called the negative-going 400, or N400, to reflect the direction of the recorded brain activity, and was considered a sign of semantic processing. In 1992, Tufts University researchers Lee Osterhout and Phillip Holcomb observed an opposite change in brain activity 600 milliseconds after subjects heard or read a sentence that violated syntax. This change, called the positive-going 600, or P600, was taken to be a hallmark of syntactic processing.
Kuperberg set out to explore the semantic side of the equation. She presented healthy subjects with two confusing sentences—“Every morning for breakfast, the boys would plant” and “Every morning for breakfast the eggs would eat”—and monitored their brain activity. The first sentence, an unlikely but possible proposition, evoked the N400 in subjects. When they read the second, impossible statement, however, subjects produced a robust P600. At first, Kuperberg did not know what to make of the difference.
Meanwhile, Sitnikova, working with Holcomb and Kuperberg, had been producing short video clips containing strange final images, such as a birthday cake being cut by a baseball bat and a man shaving with a rolling pin (below). “We went for the biggest anomalies we could imagine,” she said. Upon watching the clips, subjects exhibited the P600.
Kuperberg and Sitnikova conducted a series of functional MRI studies at MGH and found that unlikely scenarios, presented in both sentences and video clips, are processed by a rather restricted set of brain structures while the outlandish and impossible ones call into play a wider network—one that resembles the distribution of an intriguing, newly discovered set of brain cells, the mirror neurons.
The researchers concluded that the ERP and fMRI patterns might reflect the activity of two different mental mechanisms within the semantic system—a first-pass or memory-based system and an action-based system that kicks into overdrive under specific circumstances and that, in its flexibility, may bear resemblance to syntactic processing. But questions remain. The action-based system appears to be triggered by violations concerning the relationship between an action and its allowable subjects. For example, some verbs, such as eat, require that a subject be animate, a condition that an egg does not fulfill. Frame of reference may also be important. In the context of a children’s story or cartoon, our brains might not think twice about eggs eating.
“We don’t really know what triggers the reprocessing, and we don’t even know what the nature of this reprocessing is. All we know is something else happens in certain situations and not others,” said Kuperberg. “Now we’re figuring out exactly what those situations are. Hopefully, that way we may be able to tweak them when we really need to.”
