Abstract:Introduction: Perceptual representations in language comprehension were examined using sentence-picture verification tasks. However, concerns have been raised regarding the suitability of concrete pictures for representing abstract concepts compared to image-schematic diagrams. To assess the perceptual representations of spatial and abstract domains in both first language (L1) and second language (L2) processing, the study tests bilingual speakers' mental imagery on the basis of the simulation-based L1 comprehension model and proposes a simulation-based L2 comprehension model, supported by empirical evidence from an innovative sentence-diagram verification paradigm. Methods: 41 adult L1 Mandarin Chinese speakers participated in the study. 21 participants completed the Chinese sentence-diagram verification task (Experiment 1), while 20 participants completed the translation-equivalent version in L2 English (Experiment 2). Participants read a sentence [e.g., A diligent worker walked into the office (spatial sense); A strong team headed into the final (abstract sense)] at their self-paced speed, followed by a congruent (e.g., into diagram ) or incongruent diagram (e.g., out-of diagram ), and made binary judgments to verify spatial configurations between the sentence and diagram. Semantic rating tasks in both Chinese and English were also conducted to validate congruency between diagrams and sentences in both languages. Results and discussion: Results from Experiment 1 indicate overall compatibility effects on L1 Chinese processing, unaffected by directional verbs or abstractness of sense. Results from Experiment 2 reveal interference effects on L2 English processing, with interference observed only after reading sentences encoding spatial senses, not abstract senses. Aligning with previous findings using sentence-picture verification tasks, the current findings confirm the weaker mental simulation effects in L2 processing compared to L1 processing. These findings extend the existing simulation-based L1 comprehension model, provide empirical support for the proposed simulation-based L2 comprehension model, and validate the innovative sentence-diagram verification paradigm for examining image-schematic representations in spatial and abstract language processing among Chinese-English bilinguals. The paradigm holds significant potential for research on perceptual representations in processing a broader range of grammatical and semantic properties during both online and offline L1 and L2 comprehension.

Meaning above the head: combinatorial constraints on the visual vocabulary of comics

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji

The Online Processing of Only if and Even if Conditional Statements: Implications for Mental Models

What is your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing

Zooming in on visual narrative comprehension

The Lexicon of Emoji? Conventionality Modulates Processing of Emoji

Extracting Social Meaning from a Face: The Neural Substrates and Behavioral Repercussions of Mind Perception

Flexible conceptual combination: Electrophysiological correlates and consequences for associative memory

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

What does visual suffix interference tell us about spatial location in working memory?

Congruency of Separable Affix Verb Combinations Is Linearly Indexed by the N400

Framing attention in Japanese and american comics: cross-cultural differences in attentional structure

Conversational facial signals combine into compositional meanings that change the interpretation of speaker intentions

Cognitive and Affective Aspects of Verbal Humor: A Visual-World Eye-Tracking Study

Things are looking up: Vertical eye gaze in the environment affects perceptions of emotional valence in sad faces

Emotion processing in concrete and abstract words: evidence from eye fixations during reading

Distinct and shared cortical regions of the human brain activated by pictorial depictions versus verbal descriptions: an fMRI study

Putting Geometry and Function Together — Towards a Psychologically-Plausible Computational Model for Spatial Language Comprehension

Perceptual representations in L1 and L2 spatial and abstract language processing: applying an innovative sentence-diagram verification paradigm

Processing facial emojis as social information: Evidence from visual working memory for facial emojis, simple shapes, human faces, and their relations to theory of mind

Are schematic diagrams valid visual representations of concepts? Evidence from mental imagery in online processing of English prepositions