One of the key computational features of predictive models such as GPT-3 is an element known as a forward one-way predictive transformer. This triangle really connects everything together,' Schrimpf says. And then both of these are explained by the model performance on next-word prediction. 'We found that the models that predict the neural responses well also tend to best predict human behavior responses, in the form of reading times. Activity in those same models was also highly correlated with measures of human behavioral measures such as how fast people were able to read the text. They found that the best-performing next-word prediction models had activity patterns that very closely resembled those seen in the human brain. These human datasets included functional magnetic resonance (fMRI) data and intracranial electrocorticographic measurements taken in people undergoing brain surgery for epilepsy. They then compared these patterns to activity in the human brain, measured in subjects performing three language tasks: listening to stories, reading sentences one at a time, and reading sentences in which one word is revealed at a time. Other models were designed to perform different language tasks, such as filling in a blank in a sentence.Р”Р„s each model was presented with a string of words, the researchers measured the activity of the nodes that make up the network. These include a model called GPT-3 (Generative Pre-trained Transformer 3), which, given a prompt, can generate text similar to what a human would produce. The researchers analyzed 43 different language models, including several that are optimized for next-word prediction. In the new study, the MIT team used a similar approach to compare language-processing centers in the human brain with language-processing models. Research at MIT has also shown that the underlying function of visual object recognition models matches the organization of the primate visual cortex, even though those computer models were not specifically designed to mimic the brain. Over the past decade, scientists have used deep neural networks to create models of vision that can recognize objects as well as the primate brain does. These networks contain computational 'nodes' that form connections of varying strength, and layers that pass information between each other in prescribed ways. The new, high-performing next-word prediction models belong to a class of models called deep neural networks. Martin Schrimpf, an MIT graduate student who works in CBMM, is the first author of the paper. Middleton Career Development Associate Professor of Neuroscience and a member of the McGovern Institute, are the senior authors of the study, which appears this week in the Proceedings of the National Academy of Sciences. Joshua Tenenbaum, a professor of computational cognitive science at MIT and a member of CBMM and MIT's Artificial Intelligence Laboratory (CSAIL) and Evelina Fedorenko, the Frederick A. 'It's amazing that the models fit so well, and it very indirectly suggests that maybe what the human language system is doing is predicting what's going to happen next.' Rosenblith Professor of Cognitive Neuroscience, a member of MIT's McGovern Institute for Brain Research and Center for Brains, Minds, and Machines (CBMM), and an author of the new study. 'The better the model is at predicting the next word, the more closely it fits the human brain,' says Nancy Kanwisher, the Walter A. But a new study from MIT neuroscientists suggests the underlying function of these models resembles the function of language-processing centers in the human brain.Р”В¬omputer models that perform well on other types of language tasks do not show this similarity to the human brain, offering evidence that the human brain may use next-word prediction to drive language processing.

Such models were designed to optimize performance for the specific function of predicting text, without attempting to mimic anything about how the human brain performs this task or understands language. These models can not only predict the word that comes next, but also perform tasks that seem to require some degree of genuine understanding, such as question answering, document summarization, and story completion. The most recent generation of predictive language models also appears to learn something about the underlying meaning of language.

Next artificial academy game