Abstract:The generation of hierarchical structures is central to language, music and complex action. Understanding this capacity and its potential impairments requires mapping its underlying cognitive processes to the respective neuronal underpinnings. In language, left inferior frontal gyrus and left posterior temporal cortex (superior temporal sulcus/middle temporal gyrus) are considered hubs for syntactic processing. However, it is unclear whether these regions support computations specific to language or more generally support analyses of hierarchical structure. Here, we address this issue by investigating hierarchical processing in a non-linguistic task. We test the ability to represent recursive hierarchical embedding in the visual domain by contrasting a recursion task with an iteration task. The recursion task requires participants to correctly identify continuations of a hierarchy generating procedure, while the iteration task applies a serial procedure that does not generate new hierarchical levels. In a lesion-based approach, we asked 44 patients with left hemispheric chronic brain lesion to perform recursion and iteration tasks. We modelled accuracies and response times with a drift diffusion model and for each participant obtained parametric estimates for the velocity of information accumulation (drift rates) and for the amount of information accumulated before a decision (boundary separation). We then used these estimates in lesion-behaviour analyses to investigate how brain lesions affect specific aspects of recursive hierarchical embedding. We found that lesions in the posterior temporal cortex decreased drift rate in recursive hierarchical embedding, suggesting an impaired process of rule extraction from recursive structures. Moreover, lesions in inferior temporal gyrus decreased boundary separation. The latter finding does not survive conservative correction but suggests a shift in the decision criterion. As patients also participated in a grammar comprehension experiment, we performed explorative correlation-analyses and found that visual and linguistic recursive hierarchical embedding accuracies are correlated when the latter is instantiated as sentences with two nested embedding levels. While the roles of the inferior temporal gyrus and posterior temporal cortex in linguistic processes are well established, here we show that posterior temporal cortex lesions slow information accumulation (drift rate) in the visual domain. This suggests that posterior temporal cortex is essential to acquire the (knowledge) representations necessary to parse recursive hierarchical embedding in visual structures, a finding mimicking language acquisition in young children. On the contrary, inferior frontal gyrus lesions seem to affect recursive hierarchical embedding processing by interfering with more general cognitive control (boundary separation). This interesting separation of roles, rooted on a domain-general taxonomy, raises the question of whether such cognitive framing is also applicable to other domains.

Sequential processing in comprehension of hierarchical graphs

Rule-Based and Word-Level Statistics-Based Processing of Language: Insights from Neuroscience

Reading a graph is like reading a paragraph.

Parallel and Competitive Processes in Hierarchical Analysis: Perceptual Grouping and Encoding of Closure.

Seeing "what" Through "why": Evidence from Probing the Causal Structure of Hierarchical Motion.

Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension

Sentence comprehension: A parallel distributed processing approach

Hierarchical Knowledge Graphs: A Novel Information Representation for Exploratory Search Tasks

Human Learning of Hierarchical Graphs

When Graph Data Meets Multimodal: A New Paradigm for Graph Understanding and Reasoning

Construction or updating? Event model processes during visual narrative comprehension

Recursive hierarchical embedding in vision is impaired by posterior middle temporal gyrus lesions

Graph Sequential Network for Reasoning over Sequences

A Numerical Study of the Bottom-Up and Top-Down Inference Processes in And-Or Graphs.

Exploring Graph Structure Comprehension Ability of Multimodal Large Language Models: Case Studies

Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

Neural mechanisms for learning hierarchical structures of information

A Context-Enhanced Framework for Sequential Graph Reasoning

Interactions between perceptual organization based on Gestalt laws and those based on hierarchical processing