Subscribe to Events

Abstract: State-of-the-art large language models (LLMs) excel in language modeling but require an enormous amount of training and struggle to generalize on tasks that are intuitive for humans. In contrast, humans demonstrate a remarkable ability to learn and generalize from only a few examples.  Intriguingly, past work in psychology and cognitive science has investigated “natural” inductive biases that enable rapid learning and generalization in humans and animals. Despite this, such inductive biases remain largely unexplored in LLMs. To address this gap, we recently adopted two classic tasks –transitive inference and associative inference – each of which tests a single well-defined relational inductive bias that both humans and animals have been found to possess. We will discuss recent work on relational inference in artificial neural networks, followed by our ongoing work showing that large language models either fail to generalize or exhibit brittle generalization on these tasks. This is joint work with Kenneth Kay and Samuel Lippl at the Center for Theoretical Neuroscience at Columbia University.