Predicting Brain Response To Nouns
Capturing meaning in functional MRI
Thinking of a noun—a peach, train, or bird, for example—activates specific parts of the brain. Now, scientists have trained a computer to predict such activation patterns. The achievement represents a step toward understanding language processing and could one day contribute to treatments for cognitive decline.
“If we had a better model of how the brain represents language, we’d be better able to make sense of disorders like dementia,” says Tom Mitchell, PhD, a professor of computer science at Carnegie Mellon in Pittsburgh and lead author of the research published in the May 30 issue of Science.
Functional magnetic resonance imaging, or fMRI, registers changes in blood flow within peoples’ brains as they are asked to do a specific task—such as thinking of a specific word. Since 2000, Mitchell and Marcel Just, PhD, professor of psychology at Carnegie Mellon and co-director of the Pittsburgh Brain Imaging Research Center have collaborated to train a computer to produce fMRI images like those generated by humans. The training process uses two sources of data: fMRI images collected from nine people viewing 60 nouns; and a database (derived from a trillion words of text from the Internet) describing pairings of nouns and the verbs that accompany them most frequently in written text. Noun-verb pairings are the basis of language, as anyone knows who has raised a toddler, Mitchell notes.
Once trained, the computer model was able to produce a pattern of brain activity in response to words it had never before encountered with greater than 70 percent accuracy. “We now have a model that is capable of extrapolating beyond the data on which it was trained,” Mitchell says. For example, after training, the model could predict that a food noun would provoke activity in the area of the brain mediating eating sensations, the so called gustatory cortex: “peach,” for example, frequently occurs in English paired with the verb “eat.” Similarly, a noun will activate motor areas of the brain to the degree that it co-occurs with the verb, “push,” or cortical regions related to body motion to the degree that it co-occurs with “run.”
Harvard cognitive psychologist Alfonso Caramazza, PhD, cautions that the model may be imperfect. He says it fails to capture an area of the brain that is damaged in semantic dementia, one form of brain damage in which people cannot understand the meaning of words. “Our understanding of concepts, and representation of this information in the brain, is not only sensory-motor,” Caramazza says. Evolution likely has sculpted our brains to react appropriately to inanimate things that may be either potentially dangerous or pleasurable. Emotional areas of the brain respond differently to a hammer than to a dog, he points out.
“These are deep questions to which no one has the answers, so one should be cautious,” Caramazza says, adding, “I think (the Pittsburgh team) would agree, these tools are in their infancy and we are only beginning to know how to use them.”