Abstract:This article presents exploratory work in the newly emerging field of multimodal stylistics. By bringing together literary studies, linguistics and multimodal semiotics, multimodal stylisticians aim to extend the stylistic tool box by developing methodological frameworks applicable to the description and analysis of literary texts which – in addition to wording – make use of other semiotic modes such as typography, visual images, colour and layout for their meaning-making. The approach to multimodality deployed and examined is that proposed, for instance, by Kress and van Leeuwen (e. g. Reading Images. The Grammar of Visual Design, 1996; Multimodal Discourse. The Modes and Media of Contemporary Communication, 2001) and Baldry and Thibault (Multimodal Transcription and Text Analysis, 2006). According to Kress and van Leeuwen (Multimodal Discourse. The Modes and Media of Contemporary Communication, 2001, 2) »common semiotic principles operate in and across different modes«. Following this line of thought, the article sets out to examine modality as a cross-modal semiotic principle. After a brief introduction to the field of multimodal stylistics, the concept of modality is presented in its (Hallidayan) linguistic context as a verbal resource for the expression of speaker commitment by means of modal verbs, adverbials, modalised sentences and metaphorised modality. From here, the concept is transferred to the context of visual communication where it concerns the truth value of the visual representation, described at a »lexico-grammatical« level in terms of the visual modal parameters of articulation of detail, background, light, shadow, depth and colour (Kress/van Leeuwen, Reading Images. The Grammar of Visual Design, 1996). Subsequently, the concept of modality is applied to the analysis of two explicitly multimodal novels, with particular focus on the realisation of modality in visual images and typography. The texts put up for analysis are Alexander Masters' Stuart. A Life Backwards and Jonathan Safran Foer's Extremely Loud and Incredibly Close. While Kress and van Leeuwen's modal parameters prove to apply well to the analysis of modality at a lexico-grammatical level of the photographic images, analysis of the two novels reveals that modality also appears to be at play at a discourse level as a result of the multimodal interplay of the verbal and the visual. The analysis of Masters' and Foer's sporadic use of special typography, in turn, reveals that although some of Kress and van Leeuwen's modality parameters may be applicable to typography, the descriptive system is clearly less adequate in a typographic context where further work is needed before workable tools can be added to the multimodal stylistic tool kit.

Is a Picture Worth a Thousand Words? Effects of Foregrounded Multimodal and Narrative Features

Modality. Commitment, Truth Value and Reality Claims Across Modes in Multimodal Novels

Picture This: A Review of Research Relating to Narrative Processing by Moving Image Versus Language

Aesthetic appraisals of literary style and emotional intensity in narrative engagement are neurally dissociable

Reading Fictional Narratives to Improve Social and Moral Cognition: The Influence of Narrative Perspective, Transportation, and Identification

Pictures and picturing: mental imagery whilst reading illustrated novels

Exploring the affordances of text and picture stories

The influences of narrative perspective shift and scene detail on narrative semantic processing

Using Multimodal Deep Neural Networks to Disentangle Language from Visual Aesthetics

Multimodal Discourse Analysis of Interactive Environment of Film Discourse Based on Deep Learning

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

What we 'see' when we read: Visualization and vividness in reading fictional narratives

Positioning the reader: the effect of narrative point-of-view and familiarity of experience on situation model construction

Emotional prosody modulates visual mental imagery

A multimodal parallel architecture: A cognitive framework for multimodal interactions

The Influence of Mental Imagery Instructions and Personality Characteristics on Reading Experiences

Eye movements and mental imagery during reading of literary texts with different narrative styles

Imagining emotions in storyworlds: physiological narrated perception and emotional mental imagery

Expanding the scope: multimodal dimensions in aphasia discourse analysis—preliminary findings

Processing Speech and Thoughts during Silent Reading: Direct Reference Effects for Speech by Fictional Characters in Voice-Selective Auditory Cortex and a Theory-of-Mind Network

Creativity and Perception: Unveiling the Role of Cross‐Modal Audiovisual Integration