Beyond The Surface: An investigation into the role of hierarchical structure on human sentence processing using LLMs

Interest in large language models (LLMs) has exploded lately, driving speculation about their implications as models of human language processing. Language models have proven useful in isolating language-processing functions in the brain; however, debate continues regarding whether or not the best characterization of these language-processing functions employs hierarchical syntax. This study investigates this question by comparing two language models with the same underlying Transformer-XL architecture: one informed by hierarchical syntax (Transformer Grammar) and one not (Transformer-XL). Coupling these language models with human data previously collected via fMRI, results re-affirm the role of hierarchical structure in linguistic processing, implicating Broca's Area, the right Middle Temporal Gyrus, the left Temporal Pole, and the right Pre-Frontal Cortex in these aspects of processing.